feat(proofpoint-et-intelligence): migrate connector to manager-supported mode (#6768)#6784
Open
jabesq wants to merge 12 commits into
Open
feat(proofpoint-et-intelligence): migrate connector to manager-supported mode (#6768)#6784jabesq wants to merge 12 commits into
jabesq wants to merge 12 commits into
Conversation
…d settings and clean up docstrings
…and config models
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #6784 +/- ##
===========================================
- Coverage 33.27% 0.45% -32.83%
===========================================
Files 1993 1900 -93
Lines 122690 120039 -2651
===========================================
- Hits 40829 541 -40288
- Misses 81861 119498 +37637
📢 Thoughts on this report? Let us know! 🚀 New features to boost your workflow:
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR migrates the internal-enrichment/proofpoint-et-intelligence connector to a manager-supported configuration model using connectors-sdk + Pydantic settings, replacing the legacy YAML / get_config_variable approach and updating the connector to accept injected config and helper.
Changes:
- Introduce typed Pydantic settings (
ConnectorSettings) and refactor connector entrypoint/constructor to use injected config + helper. - Remove legacy config loader modules and update the API client / converter to use the new settings and helper attributes.
- Add unit tests validating settings parsing and connector/helper initialization; update metadata/config artifacts for manager support.
Reviewed changes
Copilot reviewed 24 out of 24 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| internal-enrichment/proofpoint-et-intelligence/tests/test-requirements.txt | Pin pytest version for consistent test runs. |
| internal-enrichment/proofpoint-et-intelligence/tests/test_settings.py | Add settings validation tests using connectors-sdk error handling. |
| internal-enrichment/proofpoint-et-intelligence/tests/test_main.py | Add initialization tests for settings, helper, and connector wiring. |
| internal-enrichment/proofpoint-et-intelligence/tests/conftest.py | Ensure src/ is importable during tests. |
| internal-enrichment/proofpoint-et-intelligence/src/requirements.txt | Add connectors-sdk dependency from the monorepo. |
| internal-enrichment/proofpoint-et-intelligence/src/main.py | Update entrypoint to build settings + helper and inject into connector. |
| internal-enrichment/proofpoint-et-intelligence/src/connector/settings.py | Add typed connector + connector-specific settings with Pydantic. |
| internal-enrichment/proofpoint-et-intelligence/src/connector/services/utils.py | Add overloads/TypedDict for Utils.get_now type safety. |
| internal-enrichment/proofpoint-et-intelligence/src/connector/services/converter_to_stix.py | Use helper.connect_name for author naming. |
| internal-enrichment/proofpoint-et-intelligence/src/connector/services/config_variables.py | Remove legacy YAML/env config loader. |
| internal-enrichment/proofpoint-et-intelligence/src/connector/services/client_api.py | Switch API client to typed settings and SecretStr handling. |
| internal-enrichment/proofpoint-et-intelligence/src/connector/services/init.py | Remove legacy config export from services package. |
| internal-enrichment/proofpoint-et-intelligence/src/connector/models/config_variables_models.py | Remove legacy config variable Pydantic model. |
| internal-enrichment/proofpoint-et-intelligence/src/connector/models/init.py | Drop legacy config model export. |
| internal-enrichment/proofpoint-et-intelligence/src/connector/connector.py | Refactor connector initialization + settings usage throughout processing. |
| internal-enrichment/proofpoint-et-intelligence/src/connector/init.py | Export ConnectorSettings alongside the connector class. |
| internal-enrichment/proofpoint-et-intelligence/src/config.yml.sample | Update sample YAML to match manager-supported settings. |
| internal-enrichment/proofpoint-et-intelligence/pyproject.toml | Add minimal project metadata and pytest configuration. |
| internal-enrichment/proofpoint-et-intelligence/docker-compose.yml | Update env var defaults/mappings for the new settings. |
| internal-enrichment/proofpoint-et-intelligence/.gitignore | Ignore additional local tooling files (uv.lock, .python-version). |
| internal-enrichment/proofpoint-et-intelligence/.env.sample | Add environment variable template for local configuration. |
| internal-enrichment/proofpoint-et-intelligence/metadata/connector_manifest.json | Mark connector as manager_supported: true. |
| internal-enrichment/proofpoint-et-intelligence/metadata/connector_config_schema.json | Add generated config schema from Pydantic settings. |
| internal-enrichment/proofpoint-et-intelligence/metadata/CONNECTOR_CONFIG_DOC.md | Add generated human-readable config documentation. |
Comment on lines
+37
to
40
| url = f"{self.config.api_base_url}{source_entity_type}/{entity_value}" | ||
| if target_entity_type: | ||
| url += f"/{target_entity_type}" | ||
| return url |
Comment on lines
+772
to
777
| def process_message(self, data: dict) -> str | None: | ||
| """The main process used by the connector to collect intelligence. | ||
|
|
||
| Returns: | ||
| str : messages | ||
| str: messages | ||
| """ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed changes
get_config_variable/ YAML config loading patternsettings.pywith typed Pydantic models (ConnectorSettings,ConnectorConfig) usingSecretStrfor the API key,Literalfor TLP levels, andtimedeltafor the time windowconfigandhelperinstead of instantiating them internallyconfig_variables.pyandconfig_variables_models.pymodulesdocker-compose.yml,.env.sample, andconfig.yml.samplewith correct defaults and environment variable mappingsmanager_supported: truein the connector manifest (connector-manifest.json)connector_config_schema.jsonandCONNECTOR_CONFIG_DOC.mdfrom the Pydantic settingsUtils.get_nowfor improved type safetyconverter_to_stix.pyto usehelper.connect_nameinstead ofhelper.connector.namepyproject.tomlRelated issues
Checklist
Further comments
This PR follows the standard manager-supported migration pattern: each commit corresponds to one discrete migration step (config files → Pydantic settings → package init → entry point → tests → manifest → refactor → cleanup). The connector now uses the connectors-sdk
ConnectorSettingsbase class, which enables centralized configuration management through the OpenCTI manager. All configuration is validated at startup via Pydantic, with proper typing (SecretStr,Literal,timedelta) replacing stringly-typed config lookups. The net diff is roughly neutral in line count (467 insertions, 438 deletions), reflecting a like-for-like replacement of the legacy pattern with the modern SDK approach.