Skip to content

feat: config baseline and drift check workflows - #43

Merged
leandrolerena merged 4 commits into
developfrom
feat/config-drift-workflows
Nov 10, 2026
Merged

leandrolerena merged 4 commits into
developfrom
feat/config-drift-workflows

Conversation

@remory

@remory remory commented Oct 27, 2026

Copy link
Copy Markdown

Workflows (4e0def7)

Two new workflows in scenarios/_base/workflows/, so every scenario gets them. Both run per device (seedEntity: device) through fb.base.neops.io/network_device_detect_config_drift:0.1.0.

  • config_baseline stores the running config as the config_baseline device fact (update_baseline: true).
  • config_drift_check compares the running config against that fact and leaves the fact untouched.

Both take an optional baseline_fact_key. The block normalizes secrets before storing and before comparing, which is why the baseline is written by the drift block and not by device_save_fact.

Depends on a worker image that carries the drift block. It isn't in 0.2.0-beta.7 or any published image yet. The FRR fixes it needs are in zebbra/neops-worker-sdk-py#293.

Discovery is published first (6ec85b9)

The engine's Developer Preview license caps workflow definitions at 5. Adding these two files pushed simple-lab-discovery past the cap: the files sort before it, so it was the one refused (403 LicenseLimitExceeded), and local-lab-up failed.

bootstrap/register.py now publishes DISCOVER_WORKFLOW first, so the cap refuses an extra workflow instead. The Makefile exports DISCOVER_WORKFLOW; compose and kind-lab-discover pass it to the container. The id is matched including its version. A mismatch only prints a warning, and a test now pins the Makefile and the YAML to the same version.

FRR image ships an empty vtysh.conf (e2609e5)

Without the file, every vtysh call prints % Can't open configuration file, and that line ended up in every captured show running-config, including the stored baseline.

Testing

  • make test passes (163), along with ruff, pyrefly, py39-check and doc-build.
  • Ran end to end on frr-only with the Enterprise engine and a worker built from the SDK PR: discovery, config_baseline on 10 routers, then config_drift_check. With no changes, every router reports no drift. After a hand-edited description, that router shows the expected diff.

🤖 Generated with Claude Code

Remo Ryter and others added 3 commits October 27, 2026 09:48
The engine's license caps how many workflow definitions it holds. Published
in plain file order, a scenario with more workflows than the cap had
simple-lab-discovery refused (403 LicenseLimitExceeded) because files that
sort before it took the last slots, and local-lab-up failed.

register.py now moves DISCOVER_WORKFLOW, which the Makefile exports and
compose and kind-lab-discover pass on, to the front, so the cap refuses an
extra workflow instead. An id that no document matches only warns.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
config_baseline stores each device's running configuration as the
config_baseline fact; config_drift_check compares the running configuration
against it and leaves the fact untouched. Both use
fb.base.neops.io/network_device_detect_config_drift, which normalizes
secrets before storing and comparing, so baseline and check see the same text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Without it every vtysh call prints "% Can't open configuration file
/etc/frr/vtysh.conf", and that line lands inside every `show running-config`
a worker captures, including the config_baseline fact. The file holds
vtysh's own settings, not routing config, so frr.conf and daemons stay
scenario data.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@leandrolerena

Copy link
Copy Markdown
Member

Merge recommendation: ⛔ don't merge yet

The two new workflows call fb.base.neops.io/network_device_detect_config_drift:0.1.0, and that block does not exist on GitHub yet. It is not in https://github.com/zebbra/neops-worker-sdk-py/pull/273 or https://github.com/zebbra/neops-worker-sdk-py/pull/293 (whose body says "The drift block itself is not in this PR"), and I found it on no branch of zebbra/neops-worker-sdk-py. If this merges now, every scenario ships two workflows that can only fail with "Function block not found". They would also take 2 of the Developer Preview's 5 definition slots. The other two commits stand on their own and could merge today if split out: 6ec85b9 (discovery published first) and e2609e5 (empty vtysh.conf). The workflow commit should wait until the drift block has a worker PR, so its name, version and parameters (baseline_fact_key, update_baseline) can be checked against the real Registration.

Blocking:

  • scenarios/_base/workflows/config-baseline.workflow.yaml:25 and scenarios/_base/workflows/config-drift-check.workflow.yaml:25 name a function block that no published or open-PR worker carries. Open the worker PR with network_device_detect_config_drift first (stacked on https://github.com/zebbra/neops-worker-sdk-py/pull/293), then link it here.

Notes:

  • Publishing discovery first protects the discovery definition, but local-lab-up still fails when the cap is hit. bootstrap/register.py:180 returns 1 when any document is refused, and docker compose wait lab_bootstrap (Makefile:392, under set -e) aborts before containerlab deploys. So the PR body's "local-lab-up failed" is not fixed by this change alone. If an over-cap scenario should still come up, treat a 403 LicenseLimitExceeded on a document other than DISCOVER_WORKFLOW as a warning. Otherwise, reword the body.
  • On a fresh engine, _base has 4 distinct (package, name) definitions, which fits the preview cap of 5. The cap only bites when the engine already holds other definitions, for example a persistent engine DB. Worth one sentence in the docs paragraph.
  • .env.example:59 still pins 0.2.0-beta.7, which has no drift block. When the block is published, bump the pin (or document which image to use) in the same PR, so the lab's defaults can run the workflows it ships.
  • test_the_makefile_names_the_discovery_workflow_the_base_scenario_ships pins the Makefile and YAML together by matching text. Good enough for this purpose, and it makes the version coupling the AGENTS.md change describes enforceable.

Verified locally: worktree at the PR head e2609e5 (base develop has not moved, 0 commits behind).

  • make check → ruff, ruff format, pyrefly 0 errors, 163 passed, py39-check, shell-syntax: all green.
  • Validated both new workflow YAMLs (and the discovery one) against the engine's schema/RootWorkflow.schema.json from zebbra/neops-workflow-engine@develop with jsonschema: all valid.
  • docker build devices/frr (under a throwaway tag, not neops-lab-frr:latest): built; /etc/frr/vtysh.conf exists, empty, frr:frrvty 0640; the entrypoint does not overwrite it.
  • Searched every remote branch of zebbra/neops-worker-sdk-py, including the heads of https://github.com/zebbra/neops-worker-sdk-py/pull/273 and https://github.com/zebbra/neops-worker-sdk-py/pull/293, for detect_config_drift: no match.

Not verified: no end-to-end run of the workflows; the block they need is not available to build. Not run against a live engine or devices: the shared lab and stacks belong to other sessions.

@remory remory added the pr-new-feature PR introduces new feature/s label Nov 6, 2026
@remory
remory requested a review from leandrolerena November 6, 2026 11:17
Takes develop's discovery pins (global_discover_network 0.2.0,
simple_lab_discovery 1.4.0) and keeps the branch's DISCOVER_WORKFLOW export.
Drops the branch's Makefile-vs-YAML pin test: develop's
tests/test_discovery_workflow.py already pins the two from the files
themselves, while the branch's copy hardcoded 1.3.0.
@leandrolerena
leandrolerena merged commit 7cfc480 into develop Nov 10, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-new-feature PR introduces new feature/s

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants