feat(proofpoint-et-reputation): migrate connector to manager-supported mode (#6768)#6779
Open
jabesq wants to merge 9 commits into
Open
feat(proofpoint-et-reputation): migrate connector to manager-supported mode (#6768)#6779jabesq wants to merge 9 commits into
jabesq wants to merge 9 commits into
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #6779 +/- ##
===========================================
- Coverage 33.27% 0.40% -32.88%
===========================================
Files 1993 1900 -93
Lines 122690 120062 -2628
===========================================
- Hits 40829 487 -40342
- Misses 81861 119575 +37714
📢 Thoughts on this report? Let us know! 🚀 New features to boost your workflow:
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR migrates the Proofpoint ET Reputation external-import connector to the connectors-sdk manager-supported pattern, replacing legacy config parsing with typed Pydantic settings and updating bootstrap/testing accordingly.
Changes:
- Introduces
ConnectorSettings(Pydantic, connectors-sdk) and refactors the connector to accept(config, helper)via dependency injection. - Updates the entrypoint to the manager-supported bootstrap flow and removes legacy config code.
- Adds/updates unit tests and introduces
pyproject.toml/uv.lockto support a modern dev workflow.
Reviewed changes
Copilot reviewed 21 out of 23 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| external-import/proofpoint-et-reputation/uv.lock | Adds uv lockfile for the connector’s dev environment (currently locks pytest/pytest-mock). |
| external-import/proofpoint-et-reputation/tests/test-requirements.txt | Pins pytest + pytest-mock for connector tests. |
| external-import/proofpoint-et-reputation/tests/test_settings.py | Adds settings validation tests for the new ConnectorSettings. |
| external-import/proofpoint-et-reputation/tests/test_main.py | Adds bootstrap tests for settings/helper/connector instantiation. |
| external-import/proofpoint-et-reputation/tests/conftest.py | Updates fixtures and client fixture to match new API token handling. |
| external-import/proofpoint-et-reputation/src/requirements.txt | Adds connectors-sdk dependency to runtime requirements. |
| external-import/proofpoint-et-reputation/src/main.py | Updates entrypoint to instantiate settings + helper, then inject into connector. |
| external-import/proofpoint-et-reputation/src/connector/settings.py | Introduces typed connector + connector-specific configuration models. |
| external-import/proofpoint-et-reputation/src/connector/services/utils.py | Adds overloads to Utils.get_now for better typing. |
| external-import/proofpoint-et-reputation/src/connector/services/config_variables.py | Removes legacy YAML/env config parsing implementation. |
| external-import/proofpoint-et-reputation/src/connector/services/client_api.py | Refactors API client to accept SecretStr token directly (no config object). |
| external-import/proofpoint-et-reputation/src/connector/services/init.py | Updates exports after removing legacy config service. |
| external-import/proofpoint-et-reputation/src/connector/models/config_variables_models.py | Removes legacy Pydantic config var model. |
| external-import/proofpoint-et-reputation/src/connector/models/init.py | Updates exports after removing legacy config model. |
| external-import/proofpoint-et-reputation/src/connector/connector.py | Refactors connector initialization and uses new settings fields for behavior. |
| external-import/proofpoint-et-reputation/src/connector/init.py | Updates public exports to include ConnectorSettings. |
| external-import/proofpoint-et-reputation/src/config.yml.sample | Updates sample YAML config to match the new settings structure. |
| external-import/proofpoint-et-reputation/src/init.py | Removes empty package initializer. |
| external-import/proofpoint-et-reputation/pyproject.toml | Adds pyproject (pytest config + dependency group) for modern tooling. |
| external-import/proofpoint-et-reputation/docker-compose.yml | Updates env var examples for manager-supported settings. |
| external-import/proofpoint-et-reputation/metadata/connector_manifest.json | Enables manager_supported: true. |
| external-import/proofpoint-et-reputation/metadata/connector_config_schema.json | Adds generated JSON schema for manager UI/config validation. |
| external-import/proofpoint-et-reputation/metadata/CONNECTOR_CONFIG_DOC.md | Adds generated connector config documentation. |
Comments suppressed due to low confidence (2)
external-import/proofpoint-et-reputation/src/connector/services/client_api.py:75
connector_loggerstructured context should be passed using themeta=keyword (repository convention) rather than as a positional dict argument, to avoid ambiguity and keep calls consistent with other connectors.
self.helper.connector_logger.error(
"[CONNECTOR-API] Error occurred while building the query request.",
{"error": str(e)},
)
external-import/proofpoint-et-reputation/src/connector/connector.py:276
connector_logger.debugstructured context should be passed using themeta=keyword (repository convention) rather than as a positional dict argument, to keep logging calls consistent and avoid confusion with printf-style args.
self.helper.connector_logger.debug(
"[CONNECTOR] The creation of the entity was ignored due to your configuration of the min_score variable.",
{
"collection": collection,
"min_score_config": min_score,
"entity": model.value,
"entity_score": highest_score_converted,
},
)
jabesq
force-pushed
the
feat/6768-migrate-proofpoint-et-reputation
branch
3 times, most recently
from
d6af94c to
b561d41
Compare
jabesq
force-pushed
the
feat/6768-migrate-proofpoint-et-reputation
branch
from
b561d41 to
0ecab44
Compare
Comment on lines
+101
to
+113
| def test_settings_should_raise_when_invalid_input(settings_dict, field_name): | ||
| """ | ||
| Test that `ConnectorSettings` raises on invalid input. | ||
| """ | ||
|
|
||
| class FakeConnectorSettings(ConnectorSettings): | ||
| @classmethod | ||
| def _load_config_dict(cls, _, handler) -> dict[str, Any]: | ||
| return handler(settings_dict) | ||
|
|
||
| with pytest.raises(ConfigValidationError) as err: | ||
| FakeConnectorSettings() | ||
| assert str("Error validating configuration") in str(err) |
| from pathlib import Path | ||
| from unittest.mock import MagicMock | ||
|
|
||
| sys.path.append(str(Path(__file__).parent.parent / "src")) |
Comment on lines
66
to
70
| try: | ||
| return requests.Request( | ||
| "GET", | ||
| f"{self.base_url}{self.config.extra_api_token}/reputation/{reputation_list_entity}.json", | ||
| f"{self.base_url}{self.api_token.get_secret_value()}/reputation/{reputation_list_entity}.json", | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed changes
get_config_variable/config_variables.pyconfiguration with PydanticConnectorSettingsclass (src/connector/settings.py) for type-safe, validated configuration management(config, helper)dependency injection pattern, decoupling configuration parsing from connector logicconfig_variables_models.py,config_variables.py, and emptysrc/__init__.pymain.pyentry point to use the new manager-supported bootstrap flowUtils.get_nowfor improved type safetytest_main.py,test_settings.py) and fixed existing test infrastructure (switched to pytest-mock, updated conftest fixtures)manager_supported: trueinconnector_manifest.jsonand generatedconnector_config_schema.json+CONNECTOR_CONFIG_DOC.mdpyproject.tomlwith uv build system and updatedconfig.yml.samplefor the new configuration structuredocker-compose.ymlenvironment variables to align with new settings modelRelated issues
Checklist
Further comments
This PR migrates the Proofpoint ET Reputation connector to the manager-supported pattern using the connectors-sdk. The migration follows the standard approach:
ConnectorSettingsextendingBaseConnectorSettings) replace manualget_config_variablecalls — providing validation, type coercion, and automatic schema generation.configobject and a pre-configuredhelperinstance, making it fully testable without environment manipulation.manager_supported: trueenables the platform to manage connector lifecycle, provide config UIs via the generated JSON schema, and render documentation fromCONNECTOR_CONFIG_DOC.md.The test suite validates both the settings layer (env parsing, defaults, validation) and the main entry point bootstrap logic.