CHANGELOG.md

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[Unreleased]

0.6.0

Added

• create_decision_evaluator() (also Evaluators.create_decision_evaluator) and DecisionEvaluator create and read decision-model evaluators: one typed question per evaluator, answered with probabilities. Supports boolean_probability, categorical and discrete output types; when output_type is omitted it is inferred from answer_options. If the version request is clearly rejected (400, 403, 404, 422) the new scorer is deleted; on a timeout, network error or 5xx it is kept and the raised error names its id.
• Scorer requests now send X-Galileo-Scorer-Capabilities: decision, and ScorerTypes and ModelType include decision, so decision scorers appear in list and get responses and decode.

Fixed

• Require certifi>=2026.10.19 and uv>=0.13.4 (where used) to remove trust in the compromised eMudhra certificate authorities (CVE-2026-111943). Update CI tooling and example certificate bundle pins.
• SplunkAOTracingProcessor (OpenAI Agents) now records Responses API LLM spans, the Agents SDK default, as chat messages: tool calls and tool results keep their own roles instead of one JSON user message, the output is a single assistant message, and workflow/agent spans show the latest user message and the answer text instead of "Workflow Step" and raw output_text parts. Streamed Chat Completions runs (Runner.run_streamed), whose LLM spans hold a Responses-style object, now record the assistant message instead of a JSON dump of the whole response.
• Image and file parts in OpenAI Agents user messages, on both the Responses and Chat Completions paths, are now recorded as content blocks instead of a JSON dump of the whole message, including through SplunkAOLogger(ingestion_hook=...).

0.5.0

Added

• Agent Control bridges now preserve all supported applies_to values: llm_call, tool_call, retriever_call, trace_call, and session_call.
• SplunkAOLogger.add_llm_span() and add_single_llm_span_trace() accept image_input_tokens, audio_input_tokens, audio_output_tokens, and image_output_tokens: the image/audio share of the input and output token counts. They export as the OpenTelemetry GenAI semantic-convention attributes gen_ai.usage.{image,audio}.input_tokens and gen_ai.usage.{image,audio}.output_tokens (a subset of the input/output totals, as the spec defines them) so models with per-modality pricing, such as Gemini audio input, are costed at the right rate.
• The LangChain callbacks and the Google ADK integration (splunk-ao-adk) capture this breakdown automatically when the model reports it: Gemini via ChatVertexAI or the native Google GenAI SDK. ChatGoogleGenerativeAI (langchain-google-genai) does not pass Gemini's breakdown through, so its spans keep flat-rate cost.

Fixed

• Successful @log agent/workflow calls returning None now inherit their last child's output, matching manual logger behavior. Explicit empty answers and exception handling remain distinct from missing output.
• The OpenAI Chat Completions wrapper preserves provider finish reasons for normal and streaming responses. Manual LLM logger methods accept finish_reason; enclosing agent/workflow messages retain the terminal LLM's reason when their outputs match.
• SplunkAOTracingProcessor (OpenAI Agents) no longer double-encodes Chat Completions LLM span input and output. With Chat Completions models, workflow/agent spans without input now show the user message instead of a "<Type> Step" placeholder. The Responses API path is unchanged.

0.4.0

Added

• Added the distributed-tracing extra and high-level configure_distributed_tracing() setup for Splunk AO export plus supported upstream FastAPI/Starlette, Requests, HTTPX, and aiohttp-client instrumentation. The lower-level instrument_distributed_tracing() transport helper remains available for applications that configure their provider separately.
• Explicit SDK sessions now propagate across supported services as standard W3C gen_ai.conversation.id baggage. Session selection is ambient within the current thread or async execution context and is shared by logger instances in that context. SDK routing, authentication, and application identity are not propagated.
• Added module-level get_tracing_headers() and extract_tracing_context() helpers for standard W3C traceparent and tracestate propagation. TracingMiddleware now extracts and scopes that OpenTelemetry context for Starlette/FastAPI requests.
• SDK-owned authentication, validation, and CRUD HTTP requests are suppressed from automatic application HTTP instrumentation, preventing control-plane calls from appearing as unrelated application traces.

Changed

• The temporarily retained mode="batch" and mode="distributed" values now use identical scheduled OTLP batch export and W3C propagation. Concluding an operation ends and queues it; a per-operation flush() is not required.
• Callback handlers keep a live real root during framework execution so outbound W3C context uses the same identity and visible hierarchy that is exported at commit.
• Normal LangChain, CrewAI, Google ADK, and OpenAI Agents callbacks now enqueue each completed operation into the existing BatchSpanProcessor at that operation's end callback. The deprecated ingestion_hook retains its whole-tree compatibility behavior.
• Incoming W3C sampling decisions are honored. A remote parent with its sampled flag unset continues to propagate its trace identity, but its Splunk AO descendants are not exported.
• Default logger-owned batching now honors standard OpenTelemetry OTEL_BSP_* configuration, matching caller-owned OTel paths. Explicit internal BatchConfig values remain authoritative when supplied.

Removed

• Breaking: Removed trace_id= and span_id= from SplunkAOLogger, the logger-level proprietary get_tracing_headers() method, and custom Splunk-AO-Trace-ID / Splunk-AO-Parent-ID continuation. Use the new module-level W3C helpers instead.
• Removed the obsolete distributed-only REST streaming worker and task queue; normal telemetry in both retained modes uses the existing BatchSpanProcessor path.

Fixed

• Automatic HTTP setup now imports only the requested optional instrumentors and rolls back components installed by the current call when later setup fails, avoiding partial ownership state.
• Explicit clear_session() now masks an inbound gen_ai.conversation.id for subsequent local spans and outbound propagation in the same execution context.
• Handler cleanup now releases unfinished OTel activations in nesting order, isolates malformed child telemetry from the remaining handler trace, and keeps async LangChain callback activation in the application's execution context.
• OpenAI Agents processor state is isolated per framework trace, including concurrent traces, and its public lifecycle path now has regression coverage.

0.3.0

Breaking Changes

• Local configuration directory renamed from ~/.galileo to ~/.splunk. The override environment variable is now SPLUNK_AO_HOME_DIR (previously GALILEO_HOME_DIR).

Fixed

• Agent and workflow output conversion now removes confirmed repeated input history and keeps only the last message as the terminal output; intermediate tool-call and tool-response messages are no longer included in gen_ai.output.messages for full-history spans. The trim is now gated on the prefix-match dedup check so parallel tool-call outputs (e.g. a ToolNode with multiple simultaneous calls) are never collapsed to one message when the output does not echo the input.
• gen_ai.output.messages assistant entries whose parts contain a tool_call but carry no explicit finish_reason now receive "tool_call" instead of "unknown". Explicit source finish reasons and tool-response messages are unaffected.

0.2.1 - 2026-08-07

Fixed

• Galileo multimodal URL and base64 data blocks now export as OpenTelemetry GenAI uri and blob message parts for both input and output messages while preserving modality, MIME type, and supported extension fields.

0.2.0 - 2026-08-06

Fixed

• Agent Control spans exported over OTLP now include the control discriminator and complete agent_control.* field set required for backend classification and Controls-card rendering.

Changed

• Config file renamed (HYBIM-918): The non-secret debug snapshot written to ~/.galileo/ on logout/reset is now named splunk-ao-config.json (was galileo-python-config.json). The old file can be deleted or ignored — it is never read back and has no effect on authentication or config resolution.
• Evaluator terminology alignment in docs and errors: Updated SplunkAOEvaluators docstrings, agent stream/evaluator API docstrings, and user-visible error messages to use evaluator and agent stream vocabulary following the SplunkAOMetrics → SplunkAOEvaluators rename. Enum values are documented as matching scorer labels via the legacy /scorers API paths. The public metrics= parameter name is unchanged for API compatibility. Renamed stale test_galileo_metrics_* and test_lookup_by_galileo_metrics_enum test identifiers.

0.1.1 - 2026-08-03

Removed

• Breaking: monitor_progress() job_id parameter removed (HYBIM-931). The deprecated job_id keyword argument of Experiment.monitor_progress() has been fully removed. Callers passing job_id= must remove that argument.

0.1.0 - 2026-07-31

Added

• Added deployment-aware configuration and authentication for Splunk Observability Cloud and standalone Agent Observability deployments.
• Added O11y Cloud configuration through SPLUNK_AO_REALM, SPLUNK_AO_O11Y_TOKEN, and the optional SPLUNK_AO_O11Y_API_TOKEN.
• Added native OTLP trace export for SDK handlers and standard OpenTelemetry or OpenInference instrumentations.
• Added Project and Agent Stream routing by name or ID in both OTLP request headers and OpenTelemetry Resource attributes.
• Added standard OpenTelemetry Resource detection, including a valid service.name fallback and support for OTEL_SERVICE_NAME.
• Added structured input and output capture for workflow, agent, LLM, tool, and retriever spans, including supported multimodal content.

Changed

• Completed spans are queued immediately in an OpenTelemetry BatchSpanProcessor and exported on its configured schedule.
• flush() and async_flush() now drain completed spans without concluding an active operation.
• Independent top-level decorated calls own separate traces, while nested calls remain children of their outer operation.
• Exceptions from decorated functions, synchronous generators, and asynchronous generators are re-raised after telemetry finalization.
• Reserved Agent Observability routing keys in OTEL_RESOURCE_ATTRIBUTES are removed so SDK routing configuration remains consistent with request headers.

Diagnostics

• Standard OpenTelemetry exporter logging reports transport, authentication, retry, and non-2xx HTTP failures.
• Successful HTTP responses that reject telemetry produce sanitized, rate-limited SDK error logs and a recovery log after a later accepted response.
• Exporters expose bounded, read-only receiver-acknowledgement health. Health is unknown before an acknowledgement and after ordinary transport or non-2xx failures; it is not a delivery guarantee.