feat: flatten tenant_configs table with type column by niazkhansap · Pull Request #304 · openkcm/cmk

niazkhansap · 2026-07-01T13:47:28Z

Related cleanup PR needs to be rolled out after this.

coderabbitai · 2026-07-01T13:47:36Z

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 9d344ea3-5aea-4444-be6e-a48942dd1b29

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

Walkthrough

The PR restructures TenantConfig storage from a single legacy JSON blob to typed flat rows keyed by (Key, Type), introducing LegacyTenantConfig for backward-compatible reads. TenantConfigManager gains flat-row read/write logic with legacy fallback, a TenantConfigStore interface wires into CryptoAccessDataSyncer/CryptoReconciler, authz policies grant List access, mock repo indexing is updated, and schema/data migrations backfill flat rows.

Changes

TenantConfig flat-row storage and dependent wiring

Layer / File(s)	Summary
TenantConfig model schema change `internal/model/tenantconfigs.go`	`TenantConfig` gains a `Type` primary-key field, `Value` changes from `json.RawMessage`/jsonb to `string`/text, and a new `LegacyTenantConfig` model is added for legacy JSONB reads.
TenantConfigManager flat-row read/write implementation `internal/manager/tenantconfigs.go`, `internal/manager/export_test.go`, `internal/manager/tenantconfigs_test.go`	Adds flat-row constants/keys, rewrites workflow/keystore config get/set to prefer flat rows with legacy blob fallback, adds reconstruction/parsing/writing helpers and transactional row upsert/replace helpers, removes obsolete legacy converters, and adds dual-read/round-trip/edge-case tests.
Mock repo composite-key indexing `internal/repo/mock/db.go`, `internal/repo/mock/db_test.go`, `internal/repo/mock/constant.go`	In-memory repo now indexes `TenantConfig`/`LegacyTenantConfig` by composite `(key, type)`; `Create`, `Get`, `Delete`, `Update` are updated accordingly, and tests switch `Value` to plain strings.
TenantConfigStore interface and reconciler/syncer wiring `internal/event-processor/access_data.go`, `internal/event-processor/reconciler.go`, `cmd/event-reconciler/main.go`, `test/integration/event-processor/reconciliation_test.go`	Adds a `TenantConfigStore` interface used by `CryptoAccessDataSyncer`, extends `NewCryptoReconciler` with a `tenantConfigStore` parameter, and wires a constructed `TenantConfigManager` through the CLI entrypoint and integration test setup.
Test/authz-policy wiring and repo authorization updates `internal/authz/policy_tests/event_reconciler_test.go`, `internal/event-processor/access_data_test.go`, `internal/event-processor/reconciler_test.go`, `internal/authz/repo_internal_policies.go`, `internal/authz/repo_internal_policies_test_coverage.md`	Tests construct and pass `TenantConfigManager` into reconciler/syncer constructors, seed `LegacyTenantConfig` rows with string values, and internal roles gain `RepoActionList` on `TenantConfig` (documented in the coverage markdown).
Controller and testutils TenantConfig.Value string conversions `internal/controllers/cmk/key_controller_test.go`, `internal/controllers/cmk/tenantconfigs_controller_test.go`, `internal/testutils/model.go`, `internal/manager/workflow_test.go`	Test setups switch to `model.LegacyTenantConfig` and persist marshaled JSON as strings via a consistent unmarshal/marshal round-trip pattern.
Schema and data migrations for flattened tenant configs `migrations/tenant/data/00002_repair_keystore_config_shape.go`, `migrations/tenant/data/00003_flatten_tenant_configs.go`, `migrations/tenant/data/main.go`, `migrations/tenant/schema/00014_flatten_tenant_configs.sql`, `test/db-migration/migration_test.go`	Adds migrations to repair the DEFAULT_KEYSTORE JSON shape and add `type`/`value_text` columns with composite key/indexes, backfills flat rows from legacy blobs, registers both migrations, and adds schema/data migration tests validating up/down behavior.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

niazkhansap · 2026-07-01T13:48:57Z

@coderabbitai summary

coderabbitai · 2026-07-01T13:49:03Z

✅ Action performed

Summary regeneration triggered.

jmpTeixeira02 · 2026-07-08T16:22:28Z

+UPDATE tenant_configs
+SET value = jsonb_build_object(
+        'roleManagementConfig', jsonb_build_object( -- NOSONAR
+            'localityId', value::jsonb -> 'localityId', -- NOSONAR
+            'commonName', value::jsonb -> 'commonName', -- NOSONAR
+            'accessData', value::jsonb -> 'managementAccessData'
+        ),
+        'supportedRegions', value::jsonb -> 'supportedRegions' -- NOSONAR
+    )::jsonb
+WHERE "key" = 'DEFAULT_KEYSTORE'
+  AND value::jsonb ? 'localityId'
+  AND NOT (value::jsonb ? 'roleManagementConfig');


These are data migrations, not schema ones. This is also a long running one

Nice catch moved

jmpTeixeira02 · 2026-07-08T16:23:26Z

+-- This migration flattens tenant_configs from a single JSON blob per key into
+-- (key, value, type) flat rows so individual config entries can be filtered by
+-- type/key. Data migration 00002_flatten_tenant_configs.go backfills flat rows
+-- from the legacy blobs; the legacy blobs are dropped in a future release.
+--
+-- Down drops flat rows. Since post-flatten writes target flat rows only, Down
+-- discards any changes made after the migration. Use only before traffic;
+-- restore from backup for operational rollback after production writes.
+-- +goose Up
+
+ALTER TABLE tenant_configs ADD COLUMN IF NOT EXISTS "type" varchar(255) NOT NULL DEFAULT '';
+
+ALTER TABLE tenant_configs ALTER COLUMN value TYPE text USING value::text;
+
+ALTER TABLE tenant_configs DROP CONSTRAINT IF EXISTS tenant_configs_pkey;
+ALTER TABLE tenant_configs ADD CONSTRAINT tenant_configs_pkey PRIMARY KEY ("key", "type");
+
+CREATE INDEX IF NOT EXISTS idx_tenant_configs_type ON tenant_configs ("type");
+
+-- +goose Down
+
+DROP INDEX IF EXISTS idx_tenant_configs_type;
+
+-- Flat rows hold non-JSON text and would fail the jsonb cast below.
+DELETE FROM tenant_configs WHERE length("type") > 0;
+
+ALTER TABLE tenant_configs DROP CONSTRAINT IF EXISTS tenant_configs_pkey;
+ALTER TABLE tenant_configs DROP COLUMN IF EXISTS "type";
+ALTER TABLE tenant_configs ALTER COLUMN value TYPE jsonb USING value::jsonb;
+ALTER TABLE tenant_configs ADD CONSTRAINT tenant_configs_pkey PRIMARY KEY ("key");


None of this is safe. We need to use Expand and Contract pattern. If we run it like this we break running instances

Reworked to ensure use of the Expand & Contract pattern

jmpTeixeira02 · 2026-07-08T16:24:20Z

These data migrations are also not safe. THere needs to be version/schema checks before running them as they must only be executed if the Database is in a certain state, otherwise they need to be skipped

Added guards to both migrations

niazkhansap · 2026-07-09T18:31:06Z

@coderabbitai summary

coderabbitai · 2026-07-09T18:31:13Z

✅ Action performed

Summary regeneration triggered.

Signed-off-by: Niaz Khan <niazkhansap@gmail.com>

push-tags-from-workflow Bot added the tests label Jul 1, 2026

niazkhansap requested review from MarkMcAleese, anettlippert, bkieling, davidbolet, doanhluongsap, jmpTeixeira02 and minh-nghia July 2, 2026 08:11

niazkhansap self-assigned this Jul 2, 2026

niazkhansap force-pushed the improvement/flatten-tenant-configs branch from 28c09e5 to bb8f912 Compare July 3, 2026 09:41

minh-nghia previously approved these changes Jul 6, 2026

View reviewed changes

niazkhansap dismissed minh-nghia’s stale review via cf29c9d July 7, 2026 11:43

niazkhansap force-pushed the improvement/flatten-tenant-configs branch from bb8f912 to cf29c9d Compare July 7, 2026 11:43

niazkhansap requested a review from minh-nghia July 7, 2026 14:44

jmpTeixeira02 requested changes Jul 8, 2026

View reviewed changes

niazkhansap force-pushed the improvement/flatten-tenant-configs branch 2 times, most recently from 78a6ea9 to 7b1bed4 Compare July 9, 2026 18:09

feat: flatten tenant_configs table with type column

3808e84

Signed-off-by: Niaz Khan <niazkhansap@gmail.com>

niazkhansap force-pushed the improvement/flatten-tenant-configs branch from 7b1bed4 to 3808e84 Compare July 14, 2026 11:33

niazkhansap requested a review from jmpTeixeira02 July 14, 2026 11:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: flatten tenant_configs table with type column#304

feat: flatten tenant_configs table with type column#304
niazkhansap wants to merge 1 commit into
mainfrom
improvement/flatten-tenant-configs

niazkhansap commented Jul 1, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Jul 1, 2026 •

edited

Loading

Review skipped

Uh oh!

niazkhansap commented Jul 1, 2026

Uh oh!

coderabbitai Bot commented Jul 1, 2026

Uh oh!

jmpTeixeira02 Jul 8, 2026

Uh oh!

niazkhansap Jul 9, 2026

Uh oh!

jmpTeixeira02 Jul 8, 2026

Uh oh!

niazkhansap Jul 9, 2026 •

edited

Loading

Uh oh!

jmpTeixeira02 Jul 8, 2026

Uh oh!

niazkhansap Jul 9, 2026

Uh oh!

niazkhansap commented Jul 9, 2026

Uh oh!

coderabbitai Bot commented Jul 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

niazkhansap commented Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai Bot commented Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Walkthrough

Changes

Uh oh!

niazkhansap commented Jul 1, 2026

Uh oh!

coderabbitai Bot commented Jul 1, 2026

Uh oh!

jmpTeixeira02 Jul 8, 2026

Choose a reason for hiding this comment

Uh oh!

niazkhansap Jul 9, 2026

Choose a reason for hiding this comment

Uh oh!

jmpTeixeira02 Jul 8, 2026

Choose a reason for hiding this comment

Uh oh!

niazkhansap Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jmpTeixeira02 Jul 8, 2026

Choose a reason for hiding this comment

Uh oh!

niazkhansap Jul 9, 2026

Choose a reason for hiding this comment

Uh oh!

niazkhansap commented Jul 9, 2026

Uh oh!

coderabbitai Bot commented Jul 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

niazkhansap commented Jul 1, 2026 •

edited

Loading

coderabbitai Bot commented Jul 1, 2026 •

edited

Loading

niazkhansap Jul 9, 2026 •

edited

Loading