feat(flare): migrate connector to manager-supported mode (#6759)#6788
Open
jabesq wants to merge 13 commits into
Open
feat(flare): migrate connector to manager-supported mode (#6759)#6788jabesq wants to merge 13 commits into
jabesq wants to merge 13 commits into
Conversation
jabesq
force-pushed
the
feat/6759-flare-manager-supported
branch
from
22ffe54 to
5637937
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #6788 +/- ##
===========================================
- Coverage 33.38% 0.54% -32.85%
===========================================
Files 1994 1902 -92
Lines 122846 120237 -2609
===========================================
- Hits 41017 654 -40363
- Misses 81829 119583 +37754
📢 Thoughts on this report? Let us know! 🚀 New features to boost your workflow:
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR migrates the external-import/flare connector to a manager-supported setup using connectors-sdk/Pydantic settings, updates packaging/deployment artifacts, and reshapes the test suite to validate the new lifecycle/configuration flow.
Changes:
- Introduces
ConnectorSettings/Pydantic configuration (including a deprecatedapi_base_urlshim andSecretStrforapi_key) and updates runtime wiring. - Modernizes deployment artifacts (manifest, docs, Dockerfile, compose/config samples) and removes the legacy entrypoint script.
- Relocates/expands tests into a top-level
tests/layout, adding integration-style coverage for main/bootstrap, client, and mapping logic.
Reviewed changes
Copilot reviewed 20 out of 23 changed files in this pull request and generated 18 comments.
Show a summary per file
| File | Description |
|---|---|
| external-import/flare/src/main.py | Bootstrap wiring for settings/helper/client/mapper/connector and top-level exception handling. |
| external-import/flare/src/connector/settings.py | New typed connector + Flare settings (deprecated field shim, SecretStr, new defaults). |
| external-import/flare/src/connector/connector.py | Work-unit initiation logic refactor and improved error reporting. |
| external-import/flare/src/flare_client/api_client.py | Client initialization updated for SecretStr API key and api_domain naming. |
| external-import/flare/tests/test_main.py | Adds integration tests for settings/helper/bootstrap wiring. |
| external-import/flare/tests/test_converter_to_stix.py | Adds unit tests for mapper + observable/relationship generation behavior. |
| external-import/flare/tests/test_api_client.py | Adds unit tests for client paging/filtering/action filtering and error paths. |
| external-import/flare/tests/test_connector/test_settings.py | Updates settings tests for new defaults and renamed api_domain. |
| external-import/flare/tests/test_connector/test_events.py | Expands events tests (enums + parsing helpers + param coverage). |
| external-import/flare/tests/test_connector/test_connector.py | Updates/extends connector flow tests (note: contains duplicated test blocks). |
| external-import/flare/tests/test_client/test_flare_client.py | Updates client tests for SecretStr + api_domain. |
| external-import/flare/metadata/connector_manifest.json | Sets manager_supported: true, updates title/logo/last_verified_date metadata. |
| external-import/flare/metadata/connector_config_schema.json | Adds manager/UI config JSON schema for env-var style settings. |
| external-import/flare/metadata/CONNECTOR_CONFIG_DOC.md | Adds generated human-readable configuration documentation. |
| external-import/flare/README.md | Points users to generated config documentation. |
| external-import/flare/docker-compose.yml | Updates example env vars (needs alignment with new names/defaults). |
| external-import/flare/config.yml.sample | Updates example YAML config (needs alignment with new names/defaults). |
| external-import/flare/Dockerfile | Improves caching by copying requirements first; switches to direct python main.py entrypoint. |
| external-import/flare/entrypoint.sh | Removed legacy entrypoint script. |
| external-import/flare/src/tests/test_main.py | Removes legacy test location under src/tests/. |
| external-import/flare/src/connector/tests/test_events.py | Removes legacy test location under src/connector/tests/. |
| external-import/flare/src/connector/tests/test_connector.py | Removes legacy test location under src/connector/tests/. |
jabesq
force-pushed
the
feat/6759-flare-manager-supported
branch
from
5637937 to
e04961e
Compare
jabesq
force-pushed
the
feat/6759-flare-manager-supported
branch
2 times, most recently
from
9d79a65 to
2f1094a
Compare
jabesq
force-pushed
the
feat/6759-flare-manager-supported
branch
from
2f1094a to
9b266d0
Compare
jabesq
force-pushed
the
feat/6759-flare-manager-supported
branch
from
9b266d0 to
bcc1ad3
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 20 out of 23 changed files in this pull request and generated 13 comments.
Comments suppressed due to low confidence (1)
external-import/flare/src/connector/connector.py:38
self.work_idis a per-run value but it's never reset at the start ofprocess_message(). If the connector runs multiple times in the same process, a previous run’s work id can be reused and incorrectly marked processed/failed on later runs. Also the localwork_idvariable is unused.
def process_message(self) -> None:
try:
current_state = self.helper.get_state()
Comment on lines
67
to
+71
| self.helper.connector_logger.info( | ||
| "Sync completed", | ||
| {"imported_count": imported_count}, | ||
| ) | ||
| self.helper.api.work.to_processed(work_id, message) | ||
| self.helper.api.work.to_processed(self.work_id, message) |
Comment on lines
+19
to
+22
| id: str = Field( | ||
| description="A UUID v4 to identify the connector in OpenCTI.", | ||
| default="4ca16691-f5e3-46a2-828e-a29549a8b61f", | ||
| ) |
Comment on lines
43
to
+45
| except Exception: | ||
| traceback.print_exc() | ||
| sys.exit(1) | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| main() | ||
| exit(1) |
Comment on lines
+7
to
+10
| #name: 'Flare' # optional (default: 'Flare') | ||
| #scope: 'Incident,Observable,Indicator' # optional (default: 'Incident,Observable,Indicator') | ||
| #log_level: 'info' # optional (default: 'info') | ||
| #duration_period: 'PT1H' # optional, interval in ISO-8601 format between two runs (default: 'PT1H') |
Comment on lines
+10
to
+13
| #- CONNECTOR_NAME=Flare # optional (default: 'Flare') | ||
| #- CONNECTOR_SCOPE=Incident,Observable,Indicator # optional (default: 'Incident,Observable,Indicator') | ||
| #- CONNECTOR_LOG_LEVEL=info # optional (default: 'info') | ||
| #- CONNECTOR_DURATION_PERIOD=PT1H # optional, interval in ISO-8601 format between two runs (default: 'PT1H') |
Comment on lines
+356
to
+361
| events = [{"data": {"uid": "uid-1"}}, {"data": {"uid": "uid-2"}}] | ||
|
|
||
| result = connector.process_events(iter(events), "work-123") | ||
|
|
||
| assert result == 2 | ||
| assert mock_helper.send_stix2_bundle.call_count == 2 |
Comment on lines
+202
to
+209
| @pytest.fixture | ||
| def mock_config() -> MagicMock: | ||
| config = MagicMock() | ||
| config.connector.duration_period = timedelta(hours=1) | ||
| config.flare.lookback_days = 30 | ||
| config.flare.event_types = ["stealer_log"] | ||
| config.flare.event_actions = [] | ||
| return config |
Comment on lines
+308
to
+320
| mock_helper.get_state.return_value = None | ||
| mock_flare_client.get_events.return_value = iter([]) | ||
| mock_helper.api.work.initiate_work.return_value = "work-123" | ||
|
|
||
| connector.process_message() | ||
|
|
||
| mock_helper.set_state.assert_called_once() | ||
| state_arg = mock_helper.set_state.call_args[0][0] | ||
| assert "last_run" in state_arg | ||
| assert isinstance(state_arg["last_run"], str) | ||
| mock_helper.api.work.to_processed.assert_called_once_with( | ||
| "work-123", "Sync completed. Imported 0 events." | ||
| ) |
Comment on lines
1
to
7
| FROM python:3.12-alpine | ||
| ENV CONNECTOR_TYPE=EXTERNAL_IMPORT | ||
|
|
||
| # Copy the connector | ||
| COPY src /opt/opencti-connector-flare | ||
|
|
||
| # Install Python modules | ||
| # hadolint ignore=DL3003 | ||
| RUN apk update && apk upgrade && \ | ||
| RUN apk update && \ | ||
| apk --no-cache add git build-base libmagic libffi-dev libxml2-dev libxslt-dev |
Comment on lines
+19
to
+21
| @classmethod | ||
| def _load_config_dict(cls, _, handler) -> dict[str, Any]: | ||
| return handler( |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed changes
connectors-sdkand Pydantic settings (ConnectorSettings/BaseConnectorSettings), enabling UI-driven configuration and lifecycle management.manager_supported: truein the connector manifest and recordlast_verified_date.api_base_urltoapi_domainwith aDeprecatedFieldshim for backward compatibility until 2027-06-30.api_keyasSecretStrto prevent accidental secret leakage in logs.entrypoint.sh) to align with the standardized connector image layout.tests/directory and add integration tests validating the manager-supported workflow.config.yml.sample,docker-compose.yml, andREADME.mdto reflect the new settings structure.Related issues
Checklist
Further comments
This PR performs a full manager-supported migration for the Flare connector. The approach follows the standard migration pattern:
get_config_variablecalls with a typed PydanticConnectorSettingsclass backed byconnectors-sdkbase models, enabling the platform to manage configuration via the UI.src/connector/connector.py+src/connector/settings.py).FLARE_API_BASE_URLenv var is preserved viaDeprecatedFieldso existing deployments continue to work without changes.