feat(ctm360-threatcover): add CTM360 ThreatCover TAXII 2.1 connector (#6733)

[SamuelHassine]

Proposed changes

New external-import connector for the CTM360 ThreatCover module.

ThreatCover exposes its curated indicators/observables through a TAXII 2.1 collection, so the connector:

• discovers the TAXII server from a configurable discovery URL, resolves the configured "Observables" collection by id or title, and polls it incrementally using the TAXII added_after filter and connector state, following TAXII pagination (more / next);
• imports the returned STIX 2.1 objects, applying a configurable TLP marking and attributing SDOs to a CTM360 ThreatCover author identity (SCOs keep their native form; marking-definition / identity / relationship objects are passed through untouched);
• bundles and sends the objects to OpenCTI, then records the run timestamp as the next added_after.

It is built with the standard taxii2-client library and modeled on the generic OpenCTI taxii2 connector, supporting the same authentication options (token by default, custom API-key header, or HTTP basic, plus optional client certificate and a TAXII 2.0/2.1 toggle). It is consistent with the existing CTM360 connector family (CyberBlindSpot / Cyna / HackerView), which cover the REST-based modules; ThreatCover is a distinct, TAXII-native feed.

Related issues

Closes #6733

Types of changes

• New feature (non-breaking change which adds functionality)

Checklist

• Unit tests for the client, converter and connector (patch coverage > 80% on new code)
• Code formatting (black, isort) and linting (flake8, STIX id pylint) pass
• Generated __metadata__ config schema and documentation
• Added/updated the connector README.md, config.yml.sample, docker-compose.yml and connector_manifest.json

Further comments

OpenCTI's generic TAXII2 connector can also consume this feed; this dedicated connector provides a preset, branded ThreatCover integration with sensible defaults. The TAXII discovery URL and collection id are tenant-specific and therefore configurable. Note that the /api/v2/incidents REST endpoint belongs to the separate CyberBlindspot module, not ThreatCover.

Maintainer review (independent review-and-fix)

Independent senior review of the full connector across several passes, on top of the automated review threads. Substantive fixes:

• Data integrity: when a TAXII page reports more=true without a next cursor, the client raises Ctm360ThreatcoverAPIError instead of returning a partial page; combined with finalizing the work item in error and not advancing added_after on failure, a run retries rather than permanently skipping data.
• Work lifecycle: work_id is initialized before the try and the OpenCTI work item is closed in_error on failure (no dangling "running" entries).
• TLP markings: clear maps to the OpenCTI TLP:CLEAR custom statement marking (and TLP:AMBER+STRICT is built with allow_custom and direct x_opencti_* kwargs) via a shared _statement_marking() helper, matching connectors-sdk's TLPMarking instead of the legacy stix2.TLP_WHITE.
• Converter comment: the SCO attribution comment now matches the behavior - only SDOs are attributed; SCOs are passed through in their native form (no author ref).
• Samples/docs: CONNECTOR_SCOPE / scope marked mandatory (required by the generated schema); CONNECTOR_NAME / CONNECTOR_LOG_LEVEL corrected to optional with their real defaults (CTM360 ThreatCover / error); README TOC anchor, minimum version (6.8.12), Debugging example and a "your environment" typo fixed.
• Tests: added coverage for the more-without-next abort, the failed-work finalization (state not advanced on error), and the TLP:CLEAR / AMBER+STRICT marking shapes.

Status

All CI checks are green (tests, lint/format, STIX ID linter, codecov/patch and codecov/project) and there are 0 unresolved review threads. mergeStateStatus is BLOCKED only because reviewDecision is REVIEW_REQUIRED - the PR needs one approving review from a maintainer other than me (as the author I cannot self-approve).

[codecov]

Codecov Report

❌ Patch coverage is 88.37209% with 25 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
external-import/ctm360-threatcover/src/main.py	0.00%	13 Missing ⚠️
...tcover/src/ctm360_threatcover_client/api_client.py	90.66%	7 Missing ⚠️
...port/ctm360-threatcover/src/connector/connector.py	91.07%	5 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (414c97a) and HEAD (eb4b3c3). Click for more details.

HEAD has 110 uploads less than BASE

Flag	BASE (414c97a)	HEAD (eb4b3c3)
connectors	114	4

Additional details and impacted files

@@             Coverage Diff             @@
##           master    #6734       +/-   ##
===========================================
- Coverage   31.03%    0.39%   -30.65%     
===========================================
  Files        1986     1901       -85     
  Lines      122166   119772     -2394     
===========================================
- Hits        37919      474    -37445     
- Misses      84247   119298    +35051     

Files with missing lines	Coverage Δ
...mport/ctm360-threatcover/src/connector/__init__.py	100.00% <100.00%> (ø)
...360-threatcover/src/connector/converter_to_stix.py	100.00% <100.00%> (ø)
...mport/ctm360-threatcover/src/connector/settings.py	100.00% <100.00%> (ø)
...eatcover/src/ctm360_threatcover_client/__init__.py	100.00% <100.00%> (ø)
...port/ctm360-threatcover/src/connector/connector.py	91.07% <91.07%> (ø)
...tcover/src/ctm360_threatcover_client/api_client.py	90.66% <90.66%> (ø)
external-import/ctm360-threatcover/src/main.py	0.00% <0.00%> (ø)

... and 1080 files with indirect coverage changes

📢 Thoughts on this report? Let us know!

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Copilot]

Pull request overview

Adds a new external-import connector (ctm360-threatcover) to ingest CTM360 ThreatCover observables/indicators from a TAXII 2.1 collection into OpenCTI with incremental polling via added_after and TAXII pagination.

Changes:

• Introduces a TAXII 2.1 client, STIX post-processing (author + TLP marking), and connector orchestration/state handling.
• Adds connector configuration models plus generated metadata artifacts (manifest, JSON schema, config doc) and deployment assets (Dockerfile, compose, samples, README).
• Adds unit tests for the TAXII client, converter, settings validation, and connector runtime behavior.

Reviewed changes

Copilot reviewed 23 out of 23 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
external-import/ctm360-threatcover/src/ctm360_threatcover_client/api_client.py	TAXII 2.1 client with pagination and error wrapping.
external-import/ctm360-threatcover/src/ctm360_threatcover_client/init.py	Exposes client + exception at package level.
external-import/ctm360-threatcover/src/connector/converter_to_stix.py	Applies TLP marking and author attribution to pulled STIX objects.
external-import/ctm360-threatcover/src/connector/connector.py	Main connector run loop: state handling, bundling, and OpenCTI work item usage.
external-import/ctm360-threatcover/src/connector/settings.py	Pydantic settings for OpenCTI + connector + CTM360 ThreatCover namespace.
external-import/ctm360-threatcover/src/connector/init.py	Exposes settings + connector class.
external-import/ctm360-threatcover/src/main.py	Connector entrypoint wiring settings → helper → connector.
external-import/ctm360-threatcover/src/requirements.txt	Connector runtime dependencies.
external-import/ctm360-threatcover/tests/tests_ctm360_threatcover_client/test_api_client.py	Unit tests for TAXII client pagination/filters/error behavior.
external-import/ctm360-threatcover/tests/tests_connector/test_converter.py	Unit tests for converter attribution/marking/passthrough behavior.
external-import/ctm360-threatcover/tests/tests_connector/test_connector.py	Unit tests for connector orchestration/state updates and error handling.
external-import/ctm360-threatcover/tests/tests_connector/test_settings.py	Unit tests for settings validation (valid/invalid configs).
external-import/ctm360-threatcover/tests/test_main.py	Smoke tests for settings/helper/connector instantiation.
external-import/ctm360-threatcover/tests/conftest.py	Pytest fixtures to mock heavy pycti helper dependencies.
external-import/ctm360-threatcover/tests/test-requirements.txt	Test dependency pinning.
external-import/ctm360-threatcover/README.md	Connector documentation (installation/config/deployment/behavior).
external-import/ctm360-threatcover/Dockerfile	Container build for the connector.
external-import/ctm360-threatcover/entrypoint.sh	Container entrypoint.
external-import/ctm360-threatcover/docker-compose.yml	Docker Compose deployment example.
external-import/ctm360-threatcover/config.yml.sample	Sample config for manual deployment.
external-import/ctm360-threatcover/metadata/connector_manifest.json	Connector manifest metadata (catalog/registry).
external-import/ctm360-threatcover/metadata/connector_config_schema.json	Generated env-var JSON schema for config validation/docs.
external-import/ctm360-threatcover/metadata/CONNECTOR_CONFIG_DOC.md	Generated human-readable config documentation.

[SamuelHassine]

Review and fix pass complete

Independent senior review of the full connector plus all 11 automated review threads addressed, fixes pushed in 78ebaef.

Data integrity (the important one)

• api_client.py: when a TAXII page reports more=true but omits the next cursor, the client now raises Ctm360ThreatcoverAPIError instead of returning a partial page. Previously the connector would still advance added_after and permanently skip the remaining objects.
• connector.py: work_id is initialized before the try, and both except handlers now close the work item with in_error=True. State is intentionally not advanced on error, so the run retries rather than skipping data.

TLP markings (converter_to_stix.py)

• clear now maps to the OpenCTI TLP:CLEAR custom statement marking (definition_type='statement' + x_opencti_definition='TLP:CLEAR') instead of the legacy stix2.TLP_WHITE; TLP:AMBER+STRICT is built with allow_custom=True via a shared _statement_marking() helper, matching connectors-sdk TLPMarking. (Note: generate_id('TLP','TLP:CLEAR') deterministically shares the legacy CLEAR/WHITE UUID, so the fix is about the marking shape/label, not the id.)

Samples / docs

• CONNECTOR_SCOPE / scope marked mandatory in compose + config samples (it is required in the generated schema).
• README: fixed the TOC anchor, aligned the minimum OpenCTI version with the manifest (6.8.12), fixed the Debugging logger example, and corrected the test-requirements grammar.

Tests: added coverage for the more-without-next abort, the failed-work finalization (and that state is not advanced on error), and the TLP:CLEAR / AMBER+STRICT marking shapes. 36 tests pass; black / isort / flake8 clean; patch coverage 88% (> 80% target).

CI: all checks green.

Blocker: self-approval is not possible (I am the PR author), so one maintainer approving review is still required before merge.

[Copilot]

Pull request overview

Copilot reviewed 23 out of 23 changed files in this pull request and generated 3 comments.

[SamuelHassine]

Review-and-fix pass summary

Independent senior re-review of the full external-import/ctm360-threatcover connector plus the 3 remaining open Copilot threads (commit eb4b3c3). All three were doc/comment accuracy issues; the connector code itself reviewed clean.

• converter_to_stix.py: corrected the SCO attribution comment - only SDOs (carrying a created timestamp) are attributed to the ThreatCover author; SCOs are passed through in their native form with no author ref (the code does not set x_opencti_created_by_ref), consistent with the test that asserts created_by_ref is absent for SCOs.
• README: CONNECTOR_NAME and CONNECTOR_LOG_LEVEL are now shown as optional with their real defaults (CTM360 ThreatCover / error), matching the settings defaults and the generated schema; fixed the "you environment" -> "your environment" typo.
• Independent re-review confirmed the substantive logic is sound: the TAXII pagination aborts on more=true without a next cursor, the work item is finalized in_error and added_after is not advanced on failure (so a run retries rather than skipping data), and the TLP markings use the connectors-sdk _statement_marking shape.
• Verification: black / isort / flake8 clean and all 36 unit tests pass locally.
• CI: all checks green on eb4b3c3 (tests, lint/format, STIX ID linter, codecov/patch, codecov/project). filigran/cla satisfied (organization member).
• Review threads: 0 unresolved (all 3 replied to and resolved).
• Description refreshed to the final state; title already matches the repo's Conventional Commits convention.

Remaining (non-CI) blocker: mergeStateStatus is BLOCKED only because reviewDecision is REVIEW_REQUIRED - the PR needs one approving review from a maintainer other than me. As the author I cannot self-approve, so this requires another Filigran maintainer.

[Copilot]

Pull request overview

Copilot reviewed 23 out of 23 changed files in this pull request and generated no new comments.