Ansible collection: foundata.utils

This repository contains the foundata.utils Ansible Collection: shared transport and safety mechanics for foundata's product collections. It ships module utilities and documentation fragments only. There are no modules, roles or playbooks; a product collection imports the utilities into its own modules.

Everything a product decides stays in the product collection: the resource engine, authentication, request encoding, the application status check, compatibility policy, diff projection, locator validation, the identity probe, version parsing and normalization.

The collection depends on ansible-core alone and supports the same floors as its consumers: ansible-core 2.19 or newer on the controller, Python 3.9 or newer where a module runs.

⭐ Found this useful? Support open-source and star this project:

Table of contents

• Included content
  • errors
  • deadline
  • http_client
  • Documentation fragment foundata.utils.http_client
  • exec
  • Documentation fragment foundata.utils.exec
  • Shared error codes
• Limits of the elapsed-time budget
• Using the helpers in a product module
• Dependencies
• Development and testing
• Versioning and deprecation
• Licensing, copyright
• Author information

Included content

Every helper lives in plugins/module_utils/ and is imported as ansible_collections.foundata.utils.plugins.module_utils.<name>. The supported symbols are listed below; incidental imports from other modules are not interfaces. Names with a leading underscore are internal.

Module	Supported symbols
errors	ApiError, TransportTimeout, ErrorSchema, FailsJson, fail_from, merge_error_codes, OUTCOMES, RESERVED_RESULT_KEYS
deadline	Deadline, ERROR_CODES, MAX_SECONDS
http_client	HttpClient, HttpResponse, decode_json, http_argument_spec, ERROR_CODES, HTTP_METHODS, RESERVED_HEADERS, DEFAULT_TIMEOUT_SECONDS, DEFAULT_DEADLINE_SECONDS, DEFAULT_MAX_BODY_BYTES
exec	Execution, TimeoutProfile, LocatorProfile, CommandResult, CommandError, CommandTimeout, RunsCommand, run_bounded, exec_argument_spec, ERROR_CODES, DEFAULT_DEADLINE_SECONDS, DEFAULT_MAX_OUTPUT_BYTES

From 1.0.0 on, public signatures, exceptions, defaults, error codes and fragment options are versioned interfaces: a change that breaks them needs a new major version.

A value that a module option can carry, such as the endpoint URL, a timeout, the certificate authority path, the deadline or a header value with a credential, fails as ApiError with category argument, so the module's catch boundary reports it like any other expected failure. A value only product code sets, such as the user agent, the body limit, the method or a header name, raises ValueError or TypeError: that is a programming error.

errors

The exception base, the error schema and the safe failure call.

What it guarantees:

• ApiError is the one catch boundary. A product derives its own failure type from it and catches ApiError at the module boundary; that catch also covers TransportTimeout, which a product subclass alone would miss. Each error carries a local category, a local code or None, and an HTTP status or None.
• ErrorSchema(codes_by_category=..., fallback=..., field_validators=...) is the product's public error schema. codes_by_category maps each allowed category to its allowed codes; None among them permits an error without a code. The fallback pair must itself be allowed. merge_error_codes() folds the ERROR_CODES mappings of the helpers into the product's codes, so a product cannot forget a shared code.
• Each entry of field_validators returns the value to publish for one extra result key or raises ValueError or TypeError. Validators are trusted product code: they must allowlist public values, not just coerce them, and keep their own exception text free of sensitive values.
• fail_from(module, error, schema=..., msg=..., changed=..., outcome=..., fields=...) calls fail_json with a local message, the error's category and code validated against the product's ErrorSchema (unknown pairs become the schema's fixed fallback), the HTTP status when it is one, the caller's changed and outcome, and every extra field after its validator ran. It passes exception=None and never publishes the error's text, so a product response that leaked into an exception message does not reach the result, the traceback or the log, even with ANSIBLE_DISPLAY_TRACEBACK enabled.
• A field the schema cannot publish is left out of the result: a key that is not a string, a key in RESERVED_RESULT_KEYS or with the _ansible_ prefix, a key without a validator, or a value its validator rejects with ValueError or TypeError. The category and code stay as validated, so the module still ends with changed, outcome and the failure's own code. A validator that raises anything else is a programming error and propagates with its context and cause cleared, so it cannot chain the handled error's text.
• fail_from raises TypeError for an error that is no ApiError or a changed that is not a boolean, and ValueError for an empty msg or an outcome outside OUTCOMES or in disagreement with changed. These are programming errors and do not chain the handled error.

What it does not do:

• It does not bound or redact text. Nothing in the collection publishes product text; a product parser turns product output into local codes.
• It does not decide changed or outcome. The caller tracks mutation.
• It does not check a field's name against the success result. Choose failure field names that do not appear in the module's success result with another type; a mapping on success and a list on failure under one key confuses every caller that reads both paths.

deadline

One monotonic elapsed-time budget per module operation.

What it guarantees:

• Deadline(seconds) starts a budget when created. remaining() returns the seconds left and raises TransportTimeout with code deadline_exceeded once the budget is exhausted, on every later call too.
• A non-positive, non-finite or non-numeric budget, or one above MAX_SECONDS (10^9 seconds, about 31 years), raises ApiError with category argument and code invalid_deadline, so a bad module option fails like any other expected failure. The bound keeps every derived socket, selector and wrapper timeout representable.
• No transport resets a deadline. Create one per module operation, outside any retry or polling loop, and pass the same object to every call of that operation, so that retries, polling, read-back and command execution share one budget.

What it does not do:

• It does not interrupt anything. A blocking step that ignores the budget runs to its own end; see the limits.

http_client

A bounded byte transport over ansible.module_utils.urls.open_url.

HttpClient(base_url, user_agent=..., validate_certs=True, ca_path=None, use_proxy=True, timeout=30, max_body_bytes=4 MiB) is configured once per endpoint. request(method, path, deadline=..., headers=..., body=None) sends one request and returns an HttpResponse with status, headers under lower-cased names (a repeated header is joined with a comma) and the raw body bytes. A request body is bytes or a bytearray; a bytearray is sent without a copy.

What it guarantees:

• The request stays within the endpoint: the path must be absolute, without dot segments, a fragment, a backslash, whitespace, control or non-ASCII characters (percent-encode them), and the resulting URL must keep the endpoint's origin and path prefix. A base URL with credentials, a query or a fragment is refused, and so is a host that IDNA cannot encode, such as one with an empty label.
• Certificates are validated by default; ca_path adds a private authority and must name an existing regular file; failures loading it are translated locally. validate_certs=False is never chosen automatically.
• Redirects are not followed: a 3xx response is returned like any other, so credentials never travel to another origin. netrc is not read.
• Products cannot set Accept-Encoding, User-Agent, Host, Content-Length, Transfer-Encoding or Connection. A reserved or invalid header name is a programming error and raises ValueError. A header value may hold tabs and printable ASCII only. Anything else, such as a line break or a non-ASCII character in a credential from a module option, raises ApiError with category argument and code invalid_header before any request, without the value.
• The socket inactivity timeout is the smaller of timeout and the remaining deadline; timeout must be positive, finite and at most deadline.MAX_SECONDS. The body is read with read1, with a budget check before and after every read and a byte limit. Only the identity content encoding is requested and accepted; a compressed response is refused.
• More than one Content-Length field, or a single field containing a comma list, is a transport/http_protocol failure before reading, even when all values are equal. So is a declared Content-Length literal with more digits than the limit's decimal representation, counting leading zeros. ASCII decimal literals within that digit bound may have leading zeros; a value beyond the limit is response/body_too_large. Non-ASCII digits, signs, underscores and a body shorter than its declared length are protocol failures.
• For a length-framed body, each read is capped at the remaining declared bytes, and reading stops at the declared count without waiting for EOF. Before each nonempty read, buffered bytes beyond the remaining declared length cause transport/http_protocol. A zero length completes without reading. The client closes the response and does not wait for bytes arriving after completion. Chunked bodies use chunk framing, not a concurrent length, and remain subject to the byte limit. HEAD, 1xx, 204 and 304 responses have no body; their length fields are exempt from received-body checks.
• Every response is closed, including an error status, a refused encoding, an oversized body and a failed read.
• An HTTP error status is a response, not an exception: its body is read under the same bounds and returned, so a product parser can derive an allowlisted diagnostic.
• Failures become ApiError with the categories and codes in http_client.ERROR_CODES: argument (invalid_url, invalid_timeout, invalid_ca_path, invalid_header), request (invalid_path), response (body_too_large, content_encoding, invalid_utf8, invalid_json), transport (connection_failed, tls_validation, io_error, http_protocol) and timeout (socket_timeout, deadline_exceeded, both as TransportTimeout). Translated exception messages are local and suppress the original displayed exception chain, so no URL, header or server text enters those messages.
• decode_json(body) turns a UTF-8 JSON body into Python values and raises response errors invalid_utf8 and invalid_json without the body text. An empty body is not a document: a product that answers a write without a body checks body before it decodes. The shape of the decoded value is the product's to validate. A product that publishes one code for both failures maps them itself.
• Header values never enter a repr: HttpResponse prints its status only and HttpClient its base URL only. The client stores no headers.
• http_argument_spec() returns a fresh specification of the mechanical options on every call; extending the copy affects no other module.

What it does not do:

• It does not encode requests, set authentication or product headers, decode the body, check the HTTP or the application status, retry or paginate. Those are product semantics; decode_json is a separate call for a product that speaks JSON.
• It does not keep connections open. Each request opens its own.
• It does not make the budget a hard limit; see the limits.

Documentation fragment foundata.utils.http_client

Documents url, validate_certs, ca_path, use_proxy, timeout and deadline, matching http_argument_spec(). Credential options stay with the product, which may also override an option's description, for example to name an environment variable fallback. Each consumer validates its merged documentation against its real argument specification.

exec

run_bounded(module, execution, profile=..., deadline=..., stdin=None, max_output_bytes=1 MiB, write=False) executes one prepared command on a POSIX execution host and returns a CommandResult with rc, byte stdout and stderr, deadline_expired and cleanup_triggered. Import the module under another name, so that Python's built-in exec stays usable: from ansible_collections.foundata.utils.plugins.module_utils import exec as exec_util.

What the product supplies:

• An Execution carries argv, absolute working directory, host environment additions, an effective-environment validator, host and application UIDs, and a separate application identity command. The product validates executable paths, identifiers, prefixes and option boundaries before preparing it. Execution is a frozen dataclass; build a variant, such as the same locator with another command, with dataclasses.replace.
• Command argv must be non-secret and safe for framework debug logging. Pass secrets on stdin; this helper logs neither stdin nor the captured streams.
• Stdin is bytes or None; empty bytes are refused because core does not open a pipe for them. With None the command inherits the module process's standard input, which Ansible does not define. A command or wrapper that reads or holds stdin, such as podman exec --interactive, needs bytes or must not get the option. The identity probe always runs with None.
• With write=True, identity_argv runs through the same container or wrapper as the real command and prints the application's numeric UID.
• GNU coreutils timeout must exist at the TimeoutProfile.executable absolute path (default /usr/bin/timeout).
• The product documents and enforces its policy for the environment the command inherits, through the effective-environment validator. No environment is silently removed.

What it guarantees:

• module.run_command receives an argument list, encoding=None, binary_data=True, check_rc=False, handle_exceptions=False, use_unsafe_shell=False and expand_user_and_vars=False. Literal $HOME, ${HOME} and ~ stay literal.
• A synchronous selector supervisor writes stdin without blocking, reads each stream under its own byte limit and checks the shared Deadline. Both streams are checked before decoding, parsing or returning them. Streams and execution data are omitted from repr; raw streams never become diagnostics or Ansible results from this helper. Execution repr suppression is not a framework logging guarantee.
• The selector observes expiry directly. GNU timeout is only a fallback at ceil(remaining) + 1 seconds, never zero. Command exits 124, 137 and negative signal statuses alone are not expiry. kill_after defaults to one second and is separate from the deadline. TERM grace and each of at most two reap waits use that allowance; process creation, scheduling and uninterruptible kernel work are not hard-bounded.
• The supervisor sends TERM, then KILL when needed, to the wrapper's process group. It also cleans ordinary descendants after a parent exits. A cleanup intervention is a failure, even if the parent returned zero. descendant_cleanup also reports output pipes that stay open after the grace. Their holder may be outside the process group, for example in a detached session, and may still run: the code says cleanup was needed, not that it succeeded. A detached process that closed its pipes is not detected at all.
• The environment validator sees the actual inherited environment, the module's run_command_environ_update, and the execution's additions, including core's PYTHONPATH cleanup. The layers apply in that order and a later one wins: os.environ, then run_command_environ_update, then Execution.environment; an addition can therefore blank an inherited variable. An invalid working directory or environment fails before starting a command.
• With write=True, the runner compares the host effective UID and then executes identity_argv under the same deadline. Its bounded output must equal the expected numeric application UID.
• Mechanical failures are ApiError: CommandError for execution, size, environment, identity or cleanup failures; CommandTimeout, also a TransportTimeout, for observed expiry. Errors from a supervised command carry its bounded internal result when available. Preflight failures carry no result. Original exception text is discarded. The helper never retries.
• Command errors expose write_attempted: true only when the requested write=True invocation started, not for its identity probe or a preflight failure. This is attempt metadata, not proof of mutation. A deadline exhausted before execution raises the base TransportTimeout without it.

What it does not do:

• It does not decode output or decide whether a product status is success. Nonzero exits return a CommandResult; a reviewed product parser checks the status and shape, and maps diagnostics to local codes. Raw output must never reach exit_json, fail_json, warnings, diffs or logs.
• It does not decide mutation outcome. Count completed write calls in the product through parsing and read-back. An interrupted or uncertain write uses fail_from(..., changed=True, outcome="unknown"); reconcile by identity before any retry, using the original deadline. On failure, preserve earlier attempts and include getattr(error, "write_attempted", False) for this call.
• It does not switch the module user. Use Ansible become. It does not kill detached processes, container-side processes or remote work merely because their host client stopped. Such deployments need their own cleanup qualification and must retain an unknown write outcome.

LocatorProfile declares a product's accepted fields, a product-owned prepare(locator, command) function and four required qualification records: deadline, cleanup, identity, environment. resolve() rejects a wrong kind or unused non-null fields before product validation. Core inserts null values for omitted suboptions; these are not treated as supplied fields. The prepare function rejects conflicting values and builds both command and identity prefixes. The declared forms are:

Kind	Command and identity channel	Qualification boundary
package	Absolute local binary; host UID plus an independent effective-UID probe	Local GNU timeout/process-group mechanics exercised by the integration target; product binary remains product-owned
podman	Product-validated engine prefix with explicit container and application-user options for both command and probe	Product must qualify engine version, container identity, target environment and in-container survivors after expiry
custom	Validated fixed wrapper prefix, including a documented application-user transition	Product must qualify the wrapper's identity probe, target environment, deadline propagation and cleanup; this is trusted execution, not confinement

Only the local process mechanics are qualified here. These declarations do not advertise tested product, Podman, Docker or user-switching wrappers. Their consumers supply and test the product records before use.

Documentation fragment foundata.utils.exec

Documents deadline and the exec locator dictionary, matching the fresh exec_argument_spec(). Products narrow choices and suboptions and document their supported profiles. env and target_env are no_log maps, but secrets must still use stdin because prefixes can expose environment values as process arguments. Consumers validate merged documentation and specs.

Shared error codes

Merge the used helpers' ERROR_CODES with the product schema through errors.merge_error_codes. errors adds no transport codes. The full union:

Category	Codes
argument	invalid_ca_path, invalid_deadline, invalid_environment, invalid_execution, invalid_header, invalid_locator, invalid_timeout, invalid_url
identity	host_identity, target_identity
request	invalid_path
response	body_too_large, content_encoding, invalid_command_result, invalid_json, invalid_utf8, output_too_large
timeout	deadline_exceeded, socket_timeout
transport	command_io, command_start, connection_failed, descendant_cleanup, http_protocol, io_error, tls_validation

Limits of the elapsed-time budget

The HTTP deadline is cooperative. The transport checks it before it opens a connection and before and after each body read, and derives the socket timeout from it. Everything between two checks can run past the budget:

Situation	What bounds it	Measured in the unit tests
The server accepts and sends nothing	The socket timeout, min(timeout, remaining)	0.52 s with timeout=0.5; 0.52 s with timeout=30 and a 0.5 s deadline
The response header arrives one byte at a time	Nothing. Every byte resets the inactivity timer, and the budget is first checked once the header is complete	1.94 s for a 98-byte header at 20 ms per byte against a 0.5 s deadline
The body arrives one byte at a time	The budget, checked after each read1	0.53 s against a 0.5 s deadline
The body stalls after a few bytes	The socket timeout as set when the connection was opened	0.32 s with a 0.3 s deadline and timeout=0.6
DNS resolution	Nothing in this transport. The resolver's own timeouts apply	Not measured
A slow proxy, chunk framing of a single large chunk	The socket timeout per underlying read	Not measured

The times come from tests/unit/plugins/module_utils/test_http_client.py on a loopback server; they are printed with pytest -s. A module that needs a hard limit needs the task-level timeout, which stops the controller waiting but does not stop the module process; an interrupted write then has an unknown outcome.

Using the helpers in a product module

Declare the dependency in the product collection's galaxy.yml. Use the lowest release that provides every symbol the collection imports, test that release and the newest 1.x release, and keep the upper bound below the next major version:

dependencies:
  "foundata.utils": ">=1.0.0,<2.0.0"

ansible-galaxy collection install then installs foundata.utils together with the product collection.

A product collection wraps the transport once: it sets authentication and content headers, translates the HTTP status into its own codes and decodes the body. The module tracks write attempts and calls that wrapper, never HttpClient directly. In this example only name and enabled are documented public item fields; all other response fields stay internal.

import json
from urllib.parse import quote

from ansible.module_utils.basic import AnsibleModule

from ansible_collections.foundata.utils.plugins.module_utils import (
    deadline as deadline_util,
)
from ansible_collections.foundata.utils.plugins.module_utils import (
    http_client as http_util,
)
from ansible_collections.foundata.utils.plugins.module_utils.deadline import Deadline
from ansible_collections.foundata.utils.plugins.module_utils.errors import (
    ApiError,
    ErrorSchema,
    fail_from,
    merge_error_codes,
)
from ansible_collections.foundata.utils.plugins.module_utils.http_client import (
    HttpClient,
    decode_json,
)

SCHEMA = ErrorSchema(
    codes_by_category=merge_error_codes(
        http_util.ERROR_CODES,
        deadline_util.ERROR_CODES,
        {
            "argument": {"invalid_name"},
            "auth": {"unauthenticated", "forbidden"},
            "internal": {None},
            "product": {"rejected", "unexpected_status", "not_found"},
            "response": {"invalid_item"},
            "verification": {"not_persisted"},
        },
    ),
    fallback=("internal", None),
)


class ProductError(ApiError):
    """A failure the product client classified; never carries product text."""


def item_path(name):
    if (
        not isinstance(name, str)
        or not 1 <= len(name) <= 128
        or name in {".", ".."}
        or any(ord(char) < 32 or ord(char) == 127 for char in name)
    ):
        raise ProductError(
            "Invalid item name", category="argument", code="invalid_name"
        )
    return "/api/items/" + quote(name, safe="")


def parse_item(document, name):
    if (
        not isinstance(document, dict)
        or document.get("name") != name
        or not isinstance(document.get("enabled"), bool)
    ):
        raise ProductError(
            "Invalid item response", category="response", code="invalid_item"
        )
    return {"name": document["name"], "enabled": document["enabled"]}


class Client:
    """One authenticated endpoint for one module invocation."""

    def __init__(self, params):
        self._http = HttpClient(
            params["url"],
            user_agent="foundata.example/1.0",
            validate_certs=params["validate_certs"],
            ca_path=params["ca_path"],
            use_proxy=params["use_proxy"],
            timeout=params["timeout"],
        )
        self.deadline = Deadline(params["deadline"])
        self._headers = {
            "Authorization": f"Bearer {params['api_key']}",
            "Accept": "application/json",
        }

    def call(self, method, path, body=None):
        headers = dict(self._headers)
        data = None
        if body is not None:  # Content-Type only when a body travels
            data = json.dumps(body).encode("utf-8")
            headers["Content-Type"] = "application/json"
        response = self._http.request(
            method, path, deadline=self.deadline, headers=headers, body=data
        )
        if response.status in (401, 403) or 300 <= response.status < 400:
            code = "forbidden" if response.status == 403 else "unauthenticated"
            raise ProductError(
                "Authentication failed",
                category="auth",
                code=code,
                status=response.status,
            )
        if response.status == 404:
            raise ProductError(
                "Not found", category="product", code="not_found", status=404
            )
        if response.status >= 400:
            # The product refused the request; its text stays unread.
            raise ProductError(
                "Request rejected",
                category="product",
                code="rejected",
                status=response.status,
            )
        if response.status != 200:
            raise ProductError(
                "Unexpected status",
                category="product",
                code="unexpected_status",
                status=response.status,
            )
        return decode_json(response.body)


spec = http_util.http_argument_spec()
spec["api_key"] = {"type": "str", "required": True, "no_log": True}
spec["name"] = {"type": "str", "required": True}
module = AnsibleModule(argument_spec=spec, supports_check_mode=True)
attempts = 0
try:
    client = Client(module.params)
    name = module.params["name"]
    path = item_path(name)
    current = parse_item(client.call("GET", path), name)
    if current["enabled"]:
        module.exit_json(
            changed=False, item=current, changed_fields=[], verified="observed"
        )
    if module.check_mode:
        module.exit_json(
            changed=True,
            item={**current, "enabled": True},
            changed_fields=["enabled"],
            verified="predicted",
        )
    # Retained through response parsing and read-back; never retry blindly.
    attempts += 1
    client.call("POST", path, {"enabled": True})
    # Verify the public postcondition.
    after = parse_item(client.call("GET", path), name)
    if not after["enabled"]:
        raise ProductError(
            "Change did not persist", category="verification", code="not_persisted"
        )
    module.exit_json(
        changed=True, item=after, changed_fields=["enabled"], verified="observed"
    )
except ApiError as error:
    fail_from(
        module,
        error,
        schema=SCHEMA,
        msg="Enabling the item failed",
        changed=attempts > 0,
        outcome="unknown" if attempts > 0 else "unchanged",
    )

The helper supplies these allowlisted failure fields. Ansible may add invocation metadata and a message-free call stack; from ansible-core 2.22 on, the raw module result may also carry declared no_log values in a private field for the controller. Neither the handled exception text nor raw product responses are published by fail_from.

{
  "failed": true,
  "msg": "Enabling the item failed",
  "error_category": "product",
  "error_code": "rejected",
  "http_status": 409,
  "changed": true,
  "outcome": "unknown"
}

The documentation of such a module extends the fragment:

extends_documentation_fragment:
  - foundata.utils.http_client

Dependencies

None beyond ansible-core. See dependencies in galaxy.yml.

Development and testing

DEVELOPMENT.md describes the checks, the layout, how to change a helper and the release. Each module utility's docstring states the rules it enforces and why. The collection's own tests use a loopback server and command fixtures and qualify no product; each consumer's acceptance does that against its product. To report a problem or submit a change, see CONTRIBUTING.md.

Versioning and deprecation

This project follows Semantic Versioning. Patch releases contain compatible fixes; minor releases may add compatible features or deprecations. Breaking changes, including changes to public defaults or removal of supported runtimes, require a major release.

Deprecations are announced in the changelog and relevant documentation, with migration guidance and the planned removal version. Deprecated interfaces remain available until that major release.

Licensing, copyright

Copyright (c) 2026, foundata GmbH https://foundata.com

This project is licensed under the GNU General Public License v3.0 or later (SPDX-License-Identifier: GPL-3.0-or-later), see LICENSES/GPL-3.0-or-later.txt for the full text.

The REUSE.toml file provides detailed licensing and copyright information in a human- and machine-readable format. This includes parts that may be subject to different licensing or usage terms, such as third-party components. The repository conforms to the REUSE specification. You can use reuse spdx to create an SPDX software bill of materials (SBOM).

Trademarks

Third-party trademarks used in this repository:

• Ansible® is a trademark of Red Hat, LLC, registered in the United States and other countries.

Their use here is purely descriptive and does not imply any affiliation with or endorsement by the trademark holders.

Own and licensed trademarks used in this repository:

• foundata® is a trademark of IPAM GmbH, registered in Germany and the European Union, licensed to foundata GmbH.

Author information

This project was created and is maintained by foundata.