feat(paloalto-wildfire): add WildFire enrichment connector (#6730)

[SamuelHassine]

Proposed changes

New internal-enrichment connector for Palo Alto Networks WildFire.

When a StixFile or Artifact observable is enriched, the connector:

• selects the strongest available hash (SHA-256 > SHA-1 > MD5) and queries the WildFire public API (POST /get/verdict);
• optionally, when the hash is unknown and the observable carries an uploaded file, submits it to WildFire (POST /submit/file) and polls for the verdict (opt-in via submit_unknown, disabled by default, bounded by submission_timeout, with max_file_size enforced on download);
• maps the WildFire verdict (benign / grayware / phishing / malware / command-and-control) to an OpenCTI score and a wildfire:<verdict> label;
• fetches the WildFire report (POST /get/report) to complete the file hashes and the file type (mime_type) when available;
• attaches a STIX Malware Analysis object (product = WildFire) referencing the observable, carrying the source observable's markings, with the result mapped to the STIX malware-result vocabulary.

The connector is playbook compatible and always returns the (enriched) bundle. It is modeled on the existing hybrid-analysis-sandbox connector (connectors-sdk Pydantic settings, retry/backoff client, _process_message flow).

Related issues

Closes #6730

Types of changes

• New feature (non-breaking change which adds functionality)

Checklist

• I have added tests that prove my fix is effective or that my feature works (unit tests for the client and the connector; patch coverage > 80% on new code)
• Code formatting (black, isort) and linting (flake8, STIX id pylint) pass
• Generated __metadata__ config schema and documentation
• I have added/updated the connector README.md, config.yml.sample, docker-compose.yml and connector_manifest.json

Further comments

The API base URL is configurable to support the different WildFire cloud regions and on-premise WildFire appliances. File submission/detonation of unknown samples is disabled by default (submit_unknown); set it to true to allow the connector to upload unknown samples to WildFire for analysis, otherwise it is restricted to hash-based verdict/report lookups only (the issue scopes the first version to verdict-by-hash).

Maintainer review and fix pass

A follow-up review hardened the connector and addressed the automated review threads:

• WildFire client: Retry(raise_on_status=False) so exhausted retries raise a consistent HTTPError, and every requests.RequestException (retry exhaustion, timeouts, connection errors) is wrapped as WildfireAPIError in _post and submit_file.
• Enrichment markings: the Malware Analysis object derives its markings from the source observable's OpenCTI markings (enrichment_entity["objectMarking"], resolving each TLP to its stix2 MarkingDefinition) and includes the marking objects in the bundle, so enrichment never downgrades the TLP to the default or emits a marking reference that cleanup_inconsistent_bundle would strip; it falls back to the OpenCTI TLP:CLEAR statement marking only when the observable carries none.
• Enrichment: the report file type is mapped onto the STIX File mime_type; the connector default marking is the OpenCTI TLP:CLEAR statement marking instead of stix2.TLP_WHITE.
• Submission default: submit_unknown now defaults to false (opt-in). Submission uploads the sample to a third-party service and issue feat(paloalto-wildfire): create enrichment connector for file hash verdicts #6730 scopes the first version to verdict-by-hash only, so it is now an explicit opt-in (the feature stays available via submit_unknown=true).
• Logging uses the meta= keyword for structured context throughout the client and connector.
• README: fixed the TOC anchor; aligned the minimum OpenCTI version with the manifest (6.8.12); corrected the base-connector table (CONNECTOR_TYPE = INTERNAL_ENRICHMENT, the auto config key, CONNECTOR_AUTO default false, and the optional flags to match the generated schema); rewrote the Usage section for an internal-enrichment connector; and fixed the Debugging typo and logger example.
• Tests: added assertions for the mime_type completion and the source-derived Malware Analysis markings, the non-HTTP error wrapping, and the submit_unknown opt-in default.
• Dependencies: bumped the pinned pycti to 7.260615.0 to match the current connectors-sdk, and merged current master to de-stale so the local connectors-sdk used by run_test.sh pins the same pycti and the test environment resolves consistently.

Status

All CI checks are green (tests, lint/format, STIX ID linter, codecov/patch and codecov/project) and there are 0 unresolved review threads. mergeStateStatus is BLOCKED only because reviewDecision is REVIEW_REQUIRED - the PR needs one approving review from a maintainer other than me (as the author I cannot self-approve). The branch carries a de-stale merge commit, so it should be squash-merged.

[codecov]

Codecov Report

❌ Patch coverage is 87.83383% with 41 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
...hment/paloalto-wildfire/src/connector/connector.py	90.16%	18 Missing ⚠️
internal-enrichment/paloalto-wildfire/src/main.py	0.00%	13 Missing ⚠️
...ildfire/src/paloalto_wildfire_client/api_client.py	91.66%	10 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (3e51b9e) and HEAD (1167e4a). Click for more details.

HEAD has 115 uploads less than BASE

Flag	BASE (3e51b9e)	HEAD (1167e4a)
connectors	119	4

Additional details and impacted files

@@             Coverage Diff             @@
##           master    #6731       +/-   ##
===========================================
- Coverage   32.30%    0.48%   -31.82%     
===========================================
  Files        1985     1899       -86     
  Lines      122106   119834     -2272     
===========================================
- Hits        39441      580    -38861     
- Misses      82665   119254    +36589     

Files with missing lines	Coverage Δ
...chment/paloalto-wildfire/src/connector/__init__.py	100.00% <100.00%> (ø)
...chment/paloalto-wildfire/src/connector/settings.py	100.00% <100.00%> (ø)
...-wildfire/src/paloalto_wildfire_client/__init__.py	100.00% <100.00%> (ø)
...ildfire/src/paloalto_wildfire_client/api_client.py	91.66% <91.66%> (ø)
internal-enrichment/paloalto-wildfire/src/main.py	0.00% <0.00%> (ø)
...hment/paloalto-wildfire/src/connector/connector.py	90.16% <90.16%> (ø)

... and 1119 files with indirect coverage changes

📢 Thoughts on this report? Let us know!

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Copilot]

Pull request overview

This PR introduces a new internal-enrichment connector (internal-enrichment/paloalto-wildfire) that enriches StixFile and Artifact observables by querying the Palo Alto Networks WildFire public API for hash verdicts/reports, then emitting an enriched STIX bundle (playbook compatible).

Changes:

• Adds the WildFire enrichment connector implementation (hash selection, verdict/score/label mapping, Malware Analysis object creation, TLP gating, bundle send flow).
• Adds a thin HTTP client for the WildFire API (/get/verdict, /get/report) with retry/backoff and XML parsing.
• Adds unit tests + connector packaging assets (Dockerfile, compose/config samples, metadata/schema/docs).

Reviewed changes

Copilot reviewed 23 out of 23 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
internal-enrichment/paloalto-wildfire/src/connector/connector.py	Core enrichment flow (hash lookup, verdict mapping, STIX bundle creation/sending).
internal-enrichment/paloalto-wildfire/src/connector/settings.py	Pydantic settings models for connector + WildFire config.
internal-enrichment/paloalto-wildfire/src/connector/init.py	Exposes connector/settings package entrypoints.
internal-enrichment/paloalto-wildfire/src/paloalto_wildfire_client/api_client.py	WildFire HTTP client with retries and XML parsing.
internal-enrichment/paloalto-wildfire/src/paloalto_wildfire_client/init.py	Exposes client + exception symbols.
internal-enrichment/paloalto-wildfire/src/main.py	Connector entrypoint (helper init, run, error handling).
internal-enrichment/paloalto-wildfire/src/requirements.txt	Runtime dependencies for the connector.
internal-enrichment/paloalto-wildfire/tests/conftest.py	Test fixtures + pycti helper mocking.
internal-enrichment/paloalto-wildfire/tests/test_main.py	Smoke tests for settings/helper/connector instantiation.
internal-enrichment/paloalto-wildfire/tests/test-requirements.txt	Test dependency pin for pytest.
internal-enrichment/paloalto-wildfire/tests/tests_connector/test_connector.py	Unit tests for the connector behavior and enrichment logic.
internal-enrichment/paloalto-wildfire/tests/tests_connector/test_settings.py	Unit tests for config validation.
internal-enrichment/paloalto-wildfire/tests/tests_connector/data/file_message.json	File enrichment message fixture.
internal-enrichment/paloalto-wildfire/tests/tests_connector/data/artifact_message.json	Artifact enrichment message fixture.
internal-enrichment/paloalto-wildfire/tests/tests_paloalto_wildfire_client/test_api_client.py	Unit tests for the WildFire API client.
internal-enrichment/paloalto-wildfire/README.md	Connector documentation (setup, behavior, env vars).
internal-enrichment/paloalto-wildfire/Dockerfile	Container build for the connector.
internal-enrichment/paloalto-wildfire/entrypoint.sh	Container entrypoint script.
internal-enrichment/paloalto-wildfire/docker-compose.yml	Example deployment configuration.
internal-enrichment/paloalto-wildfire/config.yml.sample	Example config.yml for manual deployment.
internal-enrichment/paloalto-wildfire/metadata/connector_manifest.json	Connector manifest metadata for the catalog.
internal-enrichment/paloalto-wildfire/metadata/connector_config_schema.json	Generated config JSON schema.
internal-enrichment/paloalto-wildfire/metadata/CONNECTOR_CONFIG_DOC.md	Generated config documentation.

[Copilot]

Pull request overview

Copilot reviewed 23 out of 23 changed files in this pull request and generated 9 comments.

[SamuelHassine]

Review and fix pass complete

Independent senior review of the full connector plus all 16 automated review threads addressed, fixes pushed in 55f0562.

WildFire client (api_client.py)

• Retry(raise_on_status=False) so exhausted retries on status_forcelist responses surface a consistent HTTPError via raise_for_status() instead of a RetryError slipping past the handler.
• _post and submit_file now wrap every requests.RequestException (retry exhaustion, timeouts, connection errors), not just HTTPError, as WildfireAPIError.

Enrichment (connector.py)

• The WildFire report file type is now applied to the STIX File mime_type (previously parsed and dropped).
• The Malware Analysis object inherits the source observable's object_marking_refs (falling back to the connector default) so enrichment never emits an unmarked / TLP-downgraded object.
• The connector default marking is the OpenCTI TLP:CLEAR statement marking instead of the legacy stix2.TLP_WHITE.
• Structured log context is passed via the meta= keyword throughout.

Docs and tests

• README: fixed the TOC anchor, aligned the minimum OpenCTI version with the manifest (6.8.12), fixed the Debugging typo and logger example.
• Tests: assert the mime_type completion and the Malware Analysis markings, cover the non-HTTP error wrapping; fixed an == True comparison. 44 tests pass; black / isort / flake8 clean; patch coverage 88.7% (> 80% target).

CI: all checks green (the earlier codecov/patch red was a transient upload-reconciliation state and is now green).

Blocker: self-approval is not possible (I am the PR author), so one maintainer approving review is still required before merge.

[Copilot]

Pull request overview

Copilot reviewed 23 out of 23 changed files in this pull request and generated 4 comments.

[SamuelHassine]

Review-and-fix pass summary

Independent senior re-review of the full internal-enrichment/paloalto-wildfire connector plus the 4 newly-opened Copilot threads.

Code/test fixes (commit 1598ca9):

• connector.py (markings): the Malware Analysis object now derives its markings from the source observable's OpenCTI markings (enrichment_entity["objectMarking"], resolving each TLP to its stix2 MarkingDefinition) instead of the STIX object_marking_refs (which the enrichment message often does not carry), and the resolved marking objects are included in the bundle. This stops the SDO from being silently downgraded to the default TLP and keeps the marking self-contained (not stripped by cleanup_inconsistent_bundle). Falls back to TLP:CLEAR only when the observable has no marking. Covered by new TLP:AMBER assertions in test_create_knowledge_file.
• settings.py (submission default): submit_unknown now defaults to false (opt-in). Submission uploads the sample to a third-party service and issue feat(paloalto-wildfire): create enrichment connector for file hash verdicts #6730 scopes the first version to verdict-by-hash only, so it is now explicit opt-in (the feature stays available via submit_unknown=true). Updated the generated schema/doc, config.yml.sample, docker-compose.yml, the README and the default test.
• README.md: corrected the base-connector table (CONNECTOR_TYPE = INTERNAL_ENRICHMENT, the auto config key, CONNECTOR_AUTO default false, and the optional flags so it matches the generated schema) and rewrote the "Usage" section for an internal-enrichment connector (runs on demand: enrichment panel, playbook, or CONNECTOR_AUTO=true).
• src/requirements.txt: bumped the pinned pycti to 7.260615.0 to match the current connectors-sdk (the support_version stays >=6.8.12, the connector's intended minimum).

Independent-review note: the WildFire client uses a urllib3 Retry adapter (not a manual retry loop) and consumes response.text on success, so it does not have the response-not-closed leak seen in the sibling stream connectors - no change was needed there. api_client logging is also already credential-safe (the API key is sent in the POST body, not the URL).

CI fix (de-stale):

• The branch was behind master, and run_test.sh installs the local connectors-sdk from the checked-out tree. The stale copy still pinned pycti==7.260609.0, which would conflict with the 7.260615.0 reinstalled from opencti master on the push-event test run. Merging current master updates the local connectors-sdk pin, so both test runs resolve consistently.

Verification:

• black / isort --profile black / flake8 --ignore=E,W clean and all 44 unit tests pass locally. CI is fully green on the new head (both Test internal-enrichment/paloalto-wildfire runs, lint/format, STIX ID linter, codecov/patch, codecov/project). filigran/cla satisfied (organization member).
• Review threads: 0 unresolved (all 4 Copilot threads replied to and resolved). Both new commits are GPG-signed.

Remaining (non-CI) blocker: mergeStateStatus is BLOCKED only because reviewDecision is REVIEW_REQUIRED - the PR needs one approving review from a Filigran maintainer other than me. As the author I cannot self-approve. Once approved, recommend squash-merge (the branch carries a de-stale merge commit).