Skip to content

Cover Alerting v2 templates in docs, validation, and asset tests - #4054

Merged
teresaromero merged 29 commits into
elastic:mainfrom
shmsr:fix/alerting-v2-template-coverage
Nov 18, 2026
Merged

teresaromero merged 29 commits into
elastic:mainfrom
shmsr:fix/alerting-v2-template-coverage

Conversation

@shmsr

@shmsr shmsr commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Generated docs read Alerting v2 names and descriptions from rule.metadata. A v2-only package states the 9.6.0 Stack floor, a classic-only package keeps the 9.2.0 sentence, and a package that publishes both states each floor separately.

  • When both engines are published, the README shows one collapsible table with ES|QL and Standard columns, matching the Fleet Alerting tab. Matching names share a row when their descriptions agree or one is empty. Different descriptions or duplicate names get one row per template, so one conflicting name doesn't change the rest of the table. A single engine keeps its existing paragraph layout below package format 3.6.0 or its collapsible Name | Description table from 3.6.0 onward.

  • Invalid v2 templates are rejected by package-spec's SVR00014 (Add semantic validator for Alerting v2 rule template structure (SVR00014) package-spec#1273), which replaces this PR's earlier local check. v2 templates require package format_version 3.7.0, the first supported by Kibana 9.6.0. Below 3.7.0 any v2 template is an error that validation.yml cannot exclude. From 3.7.0 SVR00014 checks the structure and can be excluded like other coded rules. The check does not parse ES|QL, and classic templates are not content-validated.

  • Asset loading tests expect every classic and v2 alerting_rule_template ID on Kibana 9.2.0 or later and skip those assets on earlier Kibana versions. The version check is kibana.Client.SupportsAlertRuleTemplates(), and the behavior is documented in docs/howto/asset_testing.md.

  • Single-engine tables keep the historic Name | Description columns even when every description is blank. Dual-engine tables omit Description only when every description is blank.

Descriptions preserve Markdown code-span contents, including ES|QL punctuation and backslashes. Code-span boundaries come from the Markdown parser, including escaped delimiters and code inside link labels.

Package format version and layout rules

3.6.0 refers to the integration package's format_version in manifest.yml (for example, format_version: 3.6.0). It is the package specification version. It is separate from the integration's release version and the Elastic Stack compatibility versions 9.2.0 and 9.6.0.

Templates present Package format_version Generated README layout
Classic / v1 only Below 3.6.0 Bold name followed by a description paragraph
Alerting v2 only Below 3.6.0 Bold name followed by a description paragraph
Classic / v1 only 3.6.0 or later Collapsible Name and Description table, including blank descriptions
Alerting v2 only 3.6.0 or later Collapsible Name and Description table, including blank descriptions
Both engines Below or at/above 3.6.0 Collapsible table with ES|QL / Standard availability columns; one row per template (each with a single mark) for differing nonempty descriptions or duplicate names within an engine
Neither engine Any format version No generated alert-template content or table

All collapsible tables share the same styling and accordion label. Single-engine tables always have Name and Description columns. Dual-engine tables show Description when at least one template has a nonblank description. The ES|QL and Standard columns mark which engines publish each name; a name that cannot be represented in one row expands into one row per template. Single-engine packages below package format_version: 3.6.0 retain their existing paragraph layout.

The "Alerting v2 only, below 3.6.0" row is still how the README renders, but package-spec rejects v2 templates below 3.7.0, so a valid v2 package always gets a table.

Dependency status

go.mod now requires released github.com/elastic/package-spec/v3 v3.7.1, which includes elastic/package-spec#1273 and activates SVR00014. The temporary commit pin is removed. github.com/yuin/goldmark v1.8.7 is required by that release.

CI

test/packages/parallel/alert_rule_templates_v2 pins 9.6.0-SNAPSHOT in .stack_version. The zip-install jobs install every test package on the default 9.5.6 stack, and a zip upload skips the registry's format_version filter, so Kibana 9.5.5 would reject the v2 templates. requires_newer_stack skips packages pinned to a newer stack there. test-check-packages still tests them on their pinned stack.

The latest 9.x snapshot is also tested through test-build-install-zip-9x-snapshot, currently pinned to 9.7.0-a4ce2d27-SNAPSHOT. Buildkite runs the target and provisions yq; updatecli keeps its snapshot pin current. The default-stack jobs keep their compatibility skip, which can be removed once DefaultStackVersion reaches 9.6.0.

The ZIP-file job clears stale archives before building, and its verification URL works with Bash 3.2 and prerelease package versions. Aggregate parallel-package runs dispatch each selected package separately so its .stack_version and provider settings are honored.

flowchart TD
  T["Every test/packages/*/* package"] --> Z["test-build-install-zip jobs<br/>default stack 9.5.6"]
  Z --> D{".stack_version newer than 9.5.6?"}
  D -->|no| U["Zip upload to Fleet<br/>no spec.max or conditions.kibana check"]
  D -->|"yes: alert_rule_templates_v2"| S["Skipped here<br/>test-check-packages runs it on 9.6.0-SNAPSHOT"]
Loading

Test plan

  • go build ./... and go test ./...
  • make build format lint licenser gomod update
  • README regression cases cover missing, empty, whitespace, mixed, and nonempty descriptions across classic-only, v2-only, paired, and duplicate-name layouts, below and at package format 3.6.0; single-engine rendering is checked against its existing output
  • test/packages/other/alert_rule_templates (classic only) passes elastic-package check with its README unchanged from main
  • New test/packages/parallel/alert_rule_templates_v2 (classic and v2, 9.6.0-SNAPSHOT stack) passes elastic-package check, and its asset test installs all four templates on Kibana 9.6.0-SNAPSHOT
  • All 54 elastic/integrations packages with alert rule templates (integrations febbc3c) render byte-for-byte identically to their committed READMEs
  • go test ./internal/docs/ ./internal/packages/ ./internal/kibana/ ./internal/testrunner/runners/asset/
  • With the merged package-spec Some UI improvements around current profile #1273 in go.mod, an invalid v2 sample fails validation on rule.kind, rule.schedule, and query.format, while a classic sample passes
  • With the merged package-spec, alert_rule_templates_v2 passes elastic-package check at format_version 3.7.0, fails at 3.6.0 with one "require format_version 3.7.0" error per v2 template, and an invalid v2 template at 3.7.0 fails with SVR00014
  • A v2 template whose rule is missing or not an object fails README generation with the file path
  • Installed PostgreSQL 1.32.1, MySQL 1.31.0, and Temporal OpenTelemetry 0.1.0 from the integrations repo on Kibana 9.6.0-SNAPSHOT. Fleet's Overview README matches the generated docs for each. All three ship classic templates only: PostgreSQL and MySQL use the paragraph layout, Temporal OpenTelemetry uses the collapsible Name | Description table, and all three state the 9.2.0 floor. None render ES|QL or Standard columns, because none publish an Alerting v2 template

Re-checked locally on 18 Nov 2026 for c15df85a:

  • GOTOOLCHAIN=go1.26.10 go test -count=1 ./... passed: 57 packages with tests.
  • GOTOOLCHAIN=go1.26.10 make build format lint licenser gomod update passed with 0 lint issues. Shell syntax and ShellCheck warning checks passed.
  • Both alert-template fixtures passed elastic-package check; the committed READMEs stayed unchanged.
  • Build-validation cases rejected both v2 templates at format 3.6.0, including with SVR00014 excluded. At 3.7.0, malformed rule kind/schedule/query format failed with SVR00014, and excluding that rule allowed the structural sample. Invalid ES|QL text still passed, as documented.
  • The complete v2 fixture command and classic fixture asset test passed on isolated Kibana 9.6.0-SNAPSHOT. The v2 asset results included all four expected IDs.
  • The exact new snapshot Make target built and signed all 65 eligible fixtures, installed them on 9.7.0-a4ce2d27-SNAPSHOT, and passed every installation check. Builds, signing, uploads and API checks were real; stack lifecycle commands used an already-running isolated stack on separate ports.
  • Fleet on that exact snapshot reported all four expected classic/v2 template IDs.
  • The real direct-ZIP script passed with both alert fixtures and a deliberately invalid stale ZIP, confirming cleanup before upload and correct verification URLs.
  • Regression tests cover stale archives, default/snapshot selection, batch stack pins, CI target/updatecli/tool wiring, escaped Markdown link destinations, inline code paths, and pipe escapes. The updatecli regression also checks the pre-merge SCM main Makefile, which lacks the new ZIP recipe.

The renderer fix in 31f59190 additionally passed 28 code-span boundary cases, v2-only and mixed-engine README rendering tests, an actual CLI build checked with local GFM rendering, the full Go 1.26.10 suite (57 packages), and all required checks with 0 lint issues.

After merging current main in 829a5e57, the full Go 1.26.11 suite passed (57 packages), both alert-template fixture checks passed, and all required build/format/lint/license/module/generated-file checks passed with 0 lint issues. The merge keeps Goldmark as a direct dependency and includes main's dependency and workflow updates.

Fresh integrations compatibility sweep for 090bcc6f, against integrations d8f2a8cc31:

  • All 53 classic-only packages render the same alert section as current elastic-package main and build successfully.
  • The real GCP v2 package declares format 3.6.2 and fails with both main and this PR. A disposable copy corrected to 3.7.0 builds successfully; its five real v2 metadata entries also render correctly through the README helper.
  • All 54 isolated package asset tests passed on Kibana 9.7.0-a4ce2d27-SNAPSHOT: 962 asset results, including all 427 alert-template IDs (422 classic and five v2). The GCP copy used the format correction above.
  • 196 rendering/inventory cases, 15 positive/negative CLI build cases, the full 57-package Go suite, and required checks passed with 0 lint issues. The real asset runner also rejected a deliberately omitted inventory ID, and a simulated 9.1 version produced the expected skips.
  • Fixed an additional escaped-link destination case in 090bcc6f; the integrations source checkout was not modified.

These checks verify structure, rendering and installation. They do not execute template ES|QL or prove alert firing and recovery.

Relates #3946. SVR00014 is now supplied by the released package-spec dependency.

README rendering screenshots

Historical rendering examples from 5 Nov 2026, using GitHub-flavored Markdown. Current rendering is covered by the regression tests above. Each <details> block is expanded. The documentation link is the alert-rule-templates entry in scripts/links_table.yml. Availability marks are ✅.

A v2-only table at package format 3.6.0 matches the v2-only table below except the renderer is only reachable for a package that fails SVR00014. Classic and v2 name escaping display the same text. A dual-engine table whose descriptions are only whitespace matches the no-description table.

Classic / v1 only, package format 3.6.0 or later

Name and Description columns are always present. There are no engine columns. The Stack floor is 9.2.0.

Classic only Name and Description table

Alerting v2 only, package format 3.7.0

Name and Description come from rule.metadata. The Stack floor is 9.6.0.

Alerting v2 only Name and Description table

Classic / v1 only, below package format 3.6.0

Bold name followed by a description paragraph. No table.

Classic paragraph layout below format 3.6.0

Alerting v2 only, below package format 3.6.0

The renderer still writes the paragraph layout and reads rule.metadata. Package-spec rejects v2 templates below 3.7.0, so a valid v2 package does not use this layout.

Alerting v2 paragraph layout below format 3.6.0

Both engines, including partial availability

Matching names share a row. A blank cell means that engine has no template for the name.

Both engines with partial availability

Both engines with no descriptions

The Description column is omitted when every description is missing, empty, or whitespace.

Both engines with the Description column omitted

Same name with different descriptions

The shared name becomes one row per template. The ES|QL row is first, and each row has a single mark.

Same name with different descriptions

Literal backslashes and pipes

Literal characters stay inside their cells.

Literal backslashes and pipes in one row

Duplicate names within an engine

Two classic templates and one v2 template share a name, so the table has three rows. ES|QL is first. One duplicate does not change other names.

Duplicate names expand to one row per template

One description blank, the other not

The nonempty description is kept and the name stays on one row with both marks.

Blank and nonempty descriptions share one row

Single engine with blank descriptions

The Description column stays. This is the classic table. A v2-only table does the same and states the 9.6.0 floor.

Classic table keeps a blank Description column

Names stay literal and description links stay links

Brackets, emphasis, and angle brackets in the name render as text. A Markdown link in the description stays a link. Classic and v2 display the same name.

Escaped alert name with a description link

No templates

A heading written in the README template stays, and the helper adds no sentence or table.

Heading with no generated alert templates

Committed test package READMEs

Classic fixture test/packages/other/alert_rule_templates: one Standard template, 9.2.0 floor, Name and Description columns.

Committed classic alert rule template README

test/packages/parallel/alert_rule_templates_v2: classic and v2 templates, both Stack floors, and ES|QL / Standard columns.

Committed classic and Alerting v2 README

Generated docs read v2 metadata and state the 9.6 Stack floor, and a package that publishes both engines lists each alert once with ES|QL and Standard columns. Build validation rejects invalid v2 rule shapes without parsing ES|QL, and asset tests assert every alerting_rule_template ID.

Fixes elastic#3946
@shmsr
shmsr requested a review from a team as a code owner October 12, 2026 11:33
Copilot AI balanced review requested due to automatic review settings October 12, 2026 11:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@shmsr shmsr self-assigned this Oct 12, 2026
@shmsr shmsr added the enhancement New feature or request label Oct 12, 2026
Copilot AI balanced review requested due to automatic review settings October 13, 2026 08:02

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot AI balanced review requested due to automatic review settings October 13, 2026 08:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot AI balanced review requested due to automatic review settings October 13, 2026 09:43

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@shmsr
shmsr requested a review from a team October 13, 2026 09:48

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this. I checked it against #3946, and all three gaps from the issue are fixed:

  • Docs read rule.metadata, state the 9.6.0 floor, and tell the engines apart.
  • alerting_rule_template IDs are now asserted by asset tests.
  • The issue's invalid v2 sample now fails elastic-package build with rule.kind / rule.schedule / query.format errors. On main it builds successfully.

Unit tests pass locally. However, three regressions should be fixed before merging. I've included a suggested fix for each. They're sketches against this branch, and I haven't compiled them.

1. Valid classic templates with "engine": "v1" now fail the build

validateAlertRuleTemplate returns unsupported alert template engine %q for any engine other than "" or "v2" (internal/validation/alert_rule_template.go L114-L118). Kibana treats engine: "v1" as a valid classic template:

  • buildAlertingV1RuleTemplateEngineFilter in x-pack/platform/plugins/shared/alerting/server/rules_client/common/filters.ts says: "Matches Fleet / alerting v1 rule templates: engine: "v1" or no engine field."
  • The latest saved-object schema (raw_rule_template/v4.ts) allows engine: schema.maybe(schema.string()) on the v1 layout.

To reproduce, add "engine": "v1" to test/packages/other/alert_rule_templates/kibana/alerting_rule_template/sample-rule.json. main builds it. This branch fails with invalid content found in built zip package: ... unsupported alert template engine "v1", and validation.yml can't suppress it.

Suggested fix in validateAlertRuleTemplate:

-	// Classic templates remain governed by package-spec.
-	if doc.Attributes.Engine == "" {
-		return nil
-	}
-	if doc.Attributes.Engine != "v2" {
-		return fmt.Errorf("unsupported alert template engine %q", doc.Attributes.Engine)
-	}
+	switch doc.Attributes.Engine {
+	case "", "v1":
+		// Classic templates remain governed by package-spec.
+		return nil
+	case "v2":
+	default:
+		return fmt.Errorf("unsupported alert template engine %q", doc.Attributes.Engine)
+	}

Plus a test case in TestV2TemplateStructure:

require.NoError(t, validateAlertRuleTemplate([]byte(`{"attributes":{"engine":"v1","name":"Classic"}}`)))

2. Asset tests now fail on stacks older than 9.2 for packages that ship templates

Fleet before 9.2 has no alerting_rule_template asset type. For example, KibanaAssetType in fleet/common/types/models/epm.ts on 8.19 doesn't include it. So these templates are never installed or listed in installed_kibana on those stacks. Package-spec doesn't require Kibana 9.2+ for packages that contain them.

Example: mongodb declares kibana.version: "^8.19.0 || ^9.2.1" and ships 6 templates. Integrations CI tests each package against its oldest supported version by default (prepare_stack -> oldest_supported_version in .buildkite/scripts/common.sh). With this change, its asset test on 8.19 would report 6 could not find expected asset failures.

Suggested fix in internal/testrunner/runners/asset/tester.go. Templates are reported as skipped rather than dropped, so they still show up in the results:

var alertRuleTemplateMinKibanaVersion = semver.MustParse("9.2.0")
	// In run(), before the loop over expectedAssets:
	versionInfo, err := r.kibanaClient.Version()
	if err != nil {
		return result.WithError(fmt.Errorf("cannot request Kibana version: %w", err))
	}
	kibanaVersion, err := semver.NewVersion(versionInfo.Number)
	if err != nil {
		return result.WithError(fmt.Errorf("cannot parse Kibana version %q: %w", versionInfo.Number, err))
	}
		// Inside the loop, right after rc is created:
		if e.Type == "alerting_rule_template" && kibanaVersion.LessThan(alertRuleTemplateMinKibanaVersion) {
			tr, _ := rc.WithSkip(&testrunner.SkipConfig{
				Reason: "alert rule templates require Kibana 9.2.0 or later",
			})
			results = append(results, tr[0])
			continue
		}

3. README output changes for existing single-engine packages, so elastic-package lint/check fails in integrations

In elastic/integrations, at least 18 packages use {{alertRuleTemplates}}: 7 on format 3.4.0 and 11 on 3.6.x. I ran lint with both binaries on three of them. All pass on main and fail on this branch with README.md is outdated:

  • mongodb (3.4.0): the list format changes from **Name** + description paragraph to - **\[MongoDB\] ...**. The PR description says single-engine packages below 3.6.0 keep their existing bullet lists, but the current output on main is not a bullet list. The 3.6.0 gate was added specifically to avoid this kind of breaking change.
  • aws_ec2_otel (3.6.0) and temporal_otel (3.6.2): every name changes from [AWS EC2 OTel] ... to \[AWS EC2 OTel\] ... because of the new alertRuleNameEscaper. A bracket that isn't followed by (url) or [ref] is not a link, so it doesn't need escaping. Dropping the Description column when every description is blank also changes output (see the alert_rule_templates fixture diff).

Neither change is needed to fix #3946. Suggested fix: keep main's output in the single-engine renderers, and read names through the new helpers so v2-only packages still work:

func renderAlertRuleList(builder *strings.Builder, templates []alertRuleTemplate) {
	builder.WriteString("The following alert rule templates are available:\n\n")
	for _, template := range templates {
		fmt.Fprintf(builder, "**%s**\n\n", template.displayName())
		fmt.Fprintf(builder, "%s\n\n", template.displayDescription())
	}
}

func renderAlertRuleCollapsibleTable(builder *strings.Builder, templates []alertRuleTemplate) {
	writeAlertRuleTableOpen(builder)
	builder.WriteString("| Name | Description |\n")
	builder.WriteString("|---|---|\n")
	for _, t := range templates {
		fmt.Fprintf(builder, "| %s | %s |\n",
			escapeAlertRuleTableCell(t.displayName()),
			escapeAlertRuleTableCell(t.displayDescription()))
	}
	writeAlertRuleTableClose(builder)
}

For typical text, escapeAlertRuleTableCell gives the same output as main. It only differs when a name or description contains |, \ or a carriage return, and the pipe case already breaks the table on main. alertRuleNameEscaper and the blank-Description rule can stay in the two dual-engine tables, since no existing README uses those yet. With this change:

  • test/packages/other/alert_rule_templates/docs/README.md goes back to its main version.
  • The single-engine expectations in TestRenderAlertRuleTemplates, TestAlertRuleReadmeDescriptionColumns and TestAlertRuleReadmeFormatting need updating.

Minor (non-blocking): a few tests expect looser rules than Kibana main

These cases won't reject valid packages, but they assert the opposite of createRuleDataSchema:

  • grouping.fields: [] is tested as valid ("empty groups allowed"). Kibana uses z.array(...).min(1).
  • state_transition: null is tested as valid ("state_transition itself is nullable"). createRuleDataBaseSchema uses stateTransitionSchema.optional(), which does not accept null.
  • metadata.description: "" and artifacts: [] are accepted. Kibana uses .trim().min(1) and .min(1) respectively.

Suggested changes:

 // validateTemplateMetadata
-		if jsonKind(raw) != "string" || templateStringLength(description) > templateDescriptionMax {
-			problems.add("rule.metadata.description must be a string of at most %d characters", templateDescriptionMax)
+		if jsonKind(raw) != "string" || strings.TrimSpace(description) == "" || templateStringLength(description) > templateDescriptionMax {
+			problems.add("rule.metadata.description must be a non-empty string of at most %d characters", templateDescriptionMax)
 		}

 // validateTemplateGrouping
-	if len(fields) > templateGroupingMax {
-		problems.add("rule.grouping.fields must contain at most %d fields", templateGroupingMax)
+	if len(fields) == 0 || len(fields) > templateGroupingMax {
+		problems.add("rule.grouping.fields must contain 1 to %d fields", templateGroupingMax)
 	}

 // validateTemplateArtifacts
-	if len(artifacts) > templateArtifactsMax {
-		problems.add("rule.artifacts must contain at most %d items", templateArtifactsMax)
+	if len(artifacts) == 0 || len(artifacts) > templateArtifactsMax {
+		problems.add("rule.artifacts must contain 1 to %d items", templateArtifactsMax)
 	}

 // validateTemplateTransition
-	// The transition object itself is nullable; its phases and thresholds are not.
-	if !jsonPresent(raw) || jsonKind(raw) == "null" {
+	if !jsonPresent(raw) {
 		return
 	}

 // validateTemplateLifecycle, signal branch
-		if jsonPresent(transitionRaw) && jsonKind(transitionRaw) != "null" {
+		if jsonPresent(transitionRaw) {
 			problems.add("signal rules cannot set state_transition")
 		}

These test cases would then flip to expect errors:

  • empty groups allowed
  • nullable transition
  • the null case in TestV2SignalTransitions
  • the state_transition itself is nullable loop in TestV2TemplateOptionalNulls

@teresaromero teresaromero left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution.

However, the whole validation implementation belongs to package-spec; please move into that repository the semantic rule to validate the alert formats. Once the change is released, package spec is updated and used on elastic-package, so every build of a package will use this validation.

Related to the asset testing it belongs here. You can test end to end by pinning locally package-spec with the validation.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

validation should live as a semantic validation rule under the package-spec; once a release is done on package-spec is used by elastic-package to validate.

You can test the implementation by using a replace directive on go.mod to point into the new pacakge-spec.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, thanks!

Comment thread internal/validation/validation.go Resolved
shmsr added a commit to shmsr/package-spec that referenced this pull request Oct 13, 2026
…014)

Validate Alerting v2 rule templates under kibana/alerting_rule_template/
(metadata, schedule, query parts, lifecycle strategy, grouping, artifacts,
and state-transition values). Classic templates (engine v1 or none) are
skipped and continue to use the package specification schema.

Companion to elastic/elastic-package#4054, which removes the equivalent
local validator from elastic-package in favor of this central check.
Copilot AI balanced review requested due to automatic review settings October 13, 2026 19:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

- Single-engine tables: escape v2-only cells (pipes included) so names like
  "ES|QL" no longer split a column; classic-only output stays byte-identical.
- Dual-engine grouping: fall back per-name instead of flipping the whole table
  to a per-template layout on one conflict, and key on the normalized name so
  trailing-whitespace variants pair into one row. Drops the Engine-column
  layout in favor of the ES|QL/Standard columns for every row.
- Revert the vestigial ValidateBuiltFromZip wrapper left over from the removed
  local validator.
Copilot AI balanced review requested due to automatic review settings October 14, 2026 05:45

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@jlind23

jlind23 commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

@shmsr will this PR eventually fix #3946?

@shmsr

shmsr commented Oct 14, 2026

Copy link
Copy Markdown
Member Author

@shmsr will this PR eventually fix #3946?

Yes @jlind23!

Copilot AI balanced review requested due to automatic review settings October 14, 2026 07:28

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The parallel test runner attempts to install the package on Kibana 9.5.5 despite its 9.6.0 requirement, and stale ZIPs can bypass the build-skip compatibility guard.

2 open issues

Review effort: Balanced

@@ -0,0 +1 @@
9.6.0-SNAPSHOT

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in fda4e7f. Batch runs now test each package separately, so the v2 fixture uses its 9.6.0 stack pin. Added a regression test for this too.

Comment thread scripts/test-build-install-zip-file.sh Resolved
@shmsr
shmsr requested review from mrodm and teresaromero November 6, 2026 06:11

@teresaromero teresaromero left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, i am not approving yet as we need to release package-spec so you can remove the pin to the version. other than that lgtm

@mrodm mrodm left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still pending to release next version of package-spec.

Current changes LGTM.
Just added some suggestions for the CI scripts to be checked if they could be worthy to add.

Comment on lines +129 to +134
# Packages pinned to a newer stack are tested on that stack by test-check-packages.sh.
# Skipping the build leaves no zip to install.
if requires_newer_stack "$d" "${stackVersion}"; then
echo "--- Skipping package: ${d} (requires a stack newer than ${stackVersion})"
continue
fi

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mrodm, the fixture itself is valid (format_version: 3.7.0, ^9.6.0). It just doesn't fit the stack the zip jobs start. should we open a follow-up issue to look at how the zip-install jobs pick packages and stack versions, so we don't keep adding exceptions like this??

An option for this would be adding a new make target to test the installation of the packages with the latest 9.x SNAPSHOT.

This would involve the following steps (if I don't miss anything)

Add a new Makefile target

This ./scripts/test-build-install-zip.sh script could have a new parameter to include the stack version to run elastic-package stack up:

elastic-package/Makefile

Lines 139 to 140 in 84c460e

test-build-install-zip:
./scripts/test-build-install-zip.sh

It could be done something like:

test-build-install-zip-9x-snapshot:
	./scripts/test-build-install-zip.sh 9.7.0-c6c902e7-SNAPSHOT

I would not duplicate the other build-install scripts, with just this to test the installation in 9.x snapshot would be enough.

Add a new buildkite step for that new makefile target

echo " - label: \":go: Integration test: build-install-zip\""
echo " command: ./.buildkite/scripts/integration_tests.sh -t test-build-install-zip"
echo " agents:"
echo " provider: \"gcp\""
echo " image: \"${UBUNTU_X86_64_AGENT_IMAGE}\""
echo " artifact_paths:"
echo " - build/elastic-stack-dump/build-zip/logs/*.log"
echo " - build/packages/*.sig"

Something like:

echo "      - label: \":go: Integration test: build-install-9x-snapshot-zip\""
echo "        command: ./.buildkite/scripts/integration_tests.sh -t test-build-install-9x-snapshot-zip"
echo "        agents:"
echo "          provider: \"gcp\""
echo "          image: \"${UBUNTU_X86_64_AGENT_IMAGE}\""
echo "        artifact_paths:"
echo "          - build/elastic-stack-dump/build-zip/logs/*.log"
echo "          - build/packages/*.sig"

Update updatecli configuration

And, it would also be great to update the updatecli configuration to update automatically the 9.x SNAPSHOT of that line.

Currently, this configuration lives here and it could be added a new target to take into account that new script:

targets:
update-snapshot:
name: '[updatecli] Update latest snapshot to {{ source "latestSnapshot" }}'
kind: file
sourceid: latestSnapshot
scmid: default
spec:
file: Makefile
matchpattern: '(./scripts/test-stack-command.sh) 9\.[^\s]+-SNAPSHOT'
replacepattern: '$1 {{ source "latestSnapshot" }}-SNAPSHOT'


The skip done in these scripts about requires_newer_stack should be kept, in order to not fail those buildkite steps in the current default stack version.

WDYT @teresaromero about this approach for testing the installation using the latest snapshot?

Copilot AI balanced review requested due to automatic review settings November 18, 2026 10:37

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The new description-escaping paths alter inline ES|QL code by inserting visible backslashes, so code-span contents need to be preserved and covered by a rendering regression test.

1 open issue
1 previously missed issue

In code that hasn't changed since last review

Medium severity Preserve inline code contents when escaping Markdown

internal/​docs/​alert_rule_template.go:350

The new v2 and mixed-engine paths escape Markdown punctuation inside inline code as well as ordinary text. For example, a description containing STATS errors = COUNT(*) is emitted with COUNT(\*) inside the code span; Markdown preserves that backslash literally, so the README displays an altered ES|QL query. The same happens to braces and comparison operators. Preserve code-span contents when escaping description text, and add a rendering regression using an inline ES|QL example.

1 resolved since last review

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings November 18, 2026 10:42

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

There are no active review issues blocking approval.

1 open issue

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings November 18, 2026 11:24
@infra-vault-gh-plugin-prod

Copy link
Copy Markdown

💔 Build Failed

Failed CI Steps

History

cc @shmsr

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

There are no active review issues for the Alerting v2 template changes.

1 open issue

Review effort: Balanced

@teresaromero

Copy link
Copy Markdown
Contributor

@shmsr could you rebase this pr as now package-spec is supporting alert v2 templating? we could then move forward the release cc @mrodm

Copilot AI balanced review requested due to automatic review settings November 18, 2026 11:37

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

There are no active review issues requiring changes to the Alerting v2 documentation and asset-testing support.

1 open issue

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings November 18, 2026 12:13

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the Alerting v2 documentation and asset-testing changes.

1 open issue

Review effort: Balanced

@teresaromero
teresaromero merged commit 491687e into elastic:main Nov 18, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Alerting v2 templates have gaps in generated docs, build validation, and asset tests

6 participants