Repository navigation
RFC: Feature Flags utility with AppConfig multi-variant support #5614
Replies: 4 comments 5 replies
|
This all seems reasonable but I would like to see more information about the abstractions we might be creating to implement this. An issue with some utilities that we have moved away from is large class hierarchies and I want to make sure that we continue this trend. Regarding number 2: "Should evaluateMany on the local store validate only the named flags, or the whole document?", I think we should do the whole document. Even if the flag was not yours, we should not be processing incorrect documents. Regarding number 3: "Is a Variant Fallback acceptable, or do you want default: false to mean "disabled Variant" on Agent stores?" Could you give a code example of both behaviours so I can see a practical example? |
Design decisions and Agent verificationThe design review is complete from my side. I'm updating the RFC body to describe the current proposal rather than requiring implementers to combine the original text with this thread. This revision is ready for final review; the earlier agreement on factories and Variant fallbacks should not be read as approval of every subsequent change. Settled contracts
What the real Agent returnedI ran the official ARM64 Agent image, version
This is why missing-flag detection cannot assume 404 and why selective retrieval is not the initial batch strategy. Missing local application/environment/profile paths returned 404 instead. Basic flags have no A normal-mode Agent without AWS credentials returned 500 with Context tests confirmed that JSON-quoted, ASCII-escaped values preserve commas, quotes, equals signs, padding, empty strings, Unicode, and newlines, including multiple entries in one header. Raw commas and padding silently changed matching; percent-encoding was not decoded. Quoting keys did not preserve their names. The supported key/value subset is explicit, and array contexts are an initial API restriction rather than a claim about every possible Agent expression. The image was pinned to Python baseline and implementation checksPython 3.35.0 already shipped the missing-key, time-range, validation-cache, and empty-document warning changes proposed in the original divergence table. Python 3.36.0 warns on incompatible comparator operands but still returns a non-match; our proposal throws instead. The body now distinguishes current parity from intentional differences. Remaining work is implementation and verification: Python conformance cases, real Lambda/AppConfig integration, live IAM and service-error observations, supported Lambda-extension version mapping, bundle checks, and the repository's coverage/type checks. Local Agent tests do not establish live IAM error translation or native Lambda-extension behavior. @svozza, could you give the revised body a final review, particularly the batching strategy, checked result types, opaque-error fallback policy, and documented Python differences? |
|
Package layout change, ahead of #5906 The AppConfig stores will live on two sub-paths instead of the single
This is how |
|
Clarification on schema output, ahead of #5905 The Public API section says "Result and Fallback validation must preserve the JSON-value contract and never produce
|
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Is this related to an existing feature request or issue?
Feature request #3754
Which area does this RFC relate to?
New Feature: Feature Flags
Summary
A Feature Flags utility for Powertools for AWS Lambda (TypeScript) covering both a local rule engine over the Powertools for Python Flag Document format and AppConfig feature flags evaluated by the AppConfig Agent. One
FeatureFlagsAPI sits over both kinds of store.Retrieval reuses Parameters where a suitable provider exists, including the Agent support shipped in v2.35.0. Multi-variant retrieval uses the Agent's selective-flag and context protocol. The public contract distinguishes fallback values from Flag Defaults and makes result types and failure behavior explicit.
Use case
Two groups asked for this in #3754:
Both need explicit fallbacks when retrieval is unavailable and no
undefinedresults. Agent-only users should not pull the AppConfigData SDK client into their bundle.Proposal
Terms
defaultin the Flag Document, returned when no Rule matches.when_matchvalue. The first matching Rule wins.enabled,variant, andattributes.defaultargument, returned when the flag is missing or retrieval is unavailable. Distinct from the Flag Default.Public API
The methods are shared by every store:
evaluatereturns a resolved value or the supplied Fallback, neverundefined.evaluateManyresolves only the requested flag names and applies independent Fallbacks. It retrieves one profile/document per batch, not once per flag. Misconfiguration throws for the call rather than returning a partial result.getEnabledFeaturesreturns names whose store results haveenabled: true. It returns[]on an availability failure and throws on identifiable misconfiguration.Primitive Fallbacks on JSON-valued stores infer widened
boolean,number, orstringresult types. Successful results must match that primitive type; mismatches throw. A Fallback is not permission to coerce the retrieved value.Object, array, and null Fallbacks do not describe a reliable structured type. Without a validator, their result type is
JSONValue. An optional Standard Schema validator checks both the Fallback and successful result and supplies the structured output type:Batch evaluation applies the same primitive checks and optional validation per flag. An optional
schemasmap keyed by flag name supports typed structured batch results. The utility depends on the Standard Schema contract, not a bundled validator implementation. Result and Fallback validation must preserve the JSON-value contract and never produceundefined.Variant-backed instances have a store-aware result type: their Fallbacks must be
Variantand their results are alwaysVariant, not a narrower type inferred from a fallback literal. These requirements are checked by TypeScript and at runtime for JavaScript callers.Stores and ownership
Keep one
FeatureFlagsclass, plain store objects, and factory functions. There is no store class hierarchy. The interface sketch is:Stores own evaluation and report missing flags through
found: false.FeatureFlagsowns Fallback selection, public result checks, and logging.evaluaterequests one name throughgetMany;evaluateManyrequests the named batch;getEnabledFeaturesusesgetAll. Requested names are deduplicated before transport and response-shape selection.Factories:
ruleEngineStore({ source, now? }): reads a Flag Document fromsource: () => Promise<JSONValue>and evaluates it locally. A source can wrap any Parameters provider, S3 retrieval, or a plain function.appConfigStore(...): wraps the SDK-basedAppConfigProviderfor freeform Flag Documents.appConfigAgentStore(...): wraps ParametersgetConfigfor freeform Flag Documents retrieved through the Agent.appConfigVariantStore(...): requests AppConfig feature flags with context and returns normalized Variants, including basic flags.The SDK factory accepts the Parameters
awsSdkV3ClientandclientConfigpassthrough. Agent factories accept their relevant retrieval options, including timeout. Freeform factories support a JMESPathenvelopefor documents embedded in a larger configuration. An envelope is not a local rule engine for Agent-selected Variants.Internally, validation, compilation, comparisons, and evaluation are functions. A custom store is a plain object satisfying
Store; it must distinguish missing flags, known availability failures, and other errors.Failure contract
evaluate/evaluateManygetEnabledFeatures[][]Stores wrap known availability failures in
StoreUnavailableError. The evaluator catches that category, not every exception. Missing configurations/profiles are not missing flag keys. A structured single-flagNoSuchFlagresponse can report a missing flag; unrelated 400 responses must not be treated as missing.Parameters-backed adapters must preserve enough error metadata to apply this policy. Do not classify every
GetParameterErroras an outage or parse a human-readable error message to recover a lost status. If the existing transport loses required metadata, preserving it is an integration prerequisite, not a reason to weaken the failure contract.The Agent can conceal configuration failures behind an opaque 500. For example, an Agent without AWS credentials returned
{"Message":"Unknown internal error"}. Such responses use Fallbacks. We promise to throw for identifiable misconfiguration, not to discover causes the Agent does not expose.With a logger, a call returning Fallbacks produces one warning, including a batch where several flags are missing. An availability failure returning
[]fromgetEnabledFeaturesis also one call-level warning. No logger means no output.Local Flag Document and evaluation
Use Python's documented freeform shape:
default, optionalboolean_type, namedrules,when_match, andconditionswithaction,key, andvalue. Keep the same Action vocabulary, including theIN/KEY_IN_VALUEalias pairs, time Actions, andMODULO_RANGE.boolean_typemeanstrue, matching Python. Non-boolean flags requireboolean_type: false; do not infer it from the Flag Default."2"and"10". Do not silently sort them or attempt to recover order from an already-parsed object. Names such as"rule_10"preserve insertion order. This is a documented restriction relative to Python.isStrictEqualwhere applicable. Invalid operand combinations throw an error naming the flag, rule, key, and Action, without operand values.enableduses commonsisTruthyon the resolved value. This agrees with Python for validated JSON values: false, zero, empty strings, empty arrays/objects, and null are false; nonempty containers are true regardless of their contents. Agentenabledcomes from its explicit field.rules, with warnings rather than gratuitous new failures.Python 3.36.0 is the comparison baseline. The original RFC's divergence table included changes that have since shipped upstream:
HH:MMTrue == 1, including membership checks. TypeScript uses strict JSON types.boolean_typetrue.The schema format remains shared; identical evaluation is promised only within the documented supported semantics. AppConfig's own rule language/evaluator is not rewritten to match the local engine.
Validation cache and snapshots
Validate the whole Flag Document, even when a call requests only a subset of flags. On retrieval, compare serialized document content with the last validated document without deliberately sorting keys. If unchanged, reuse the compiled rules. If changed, validate and create an owned snapshot.
Keep only the latest validated, compiled snapshot per store. This is not a retrieval cache and does not cache evaluation results by context. Equivalent fresh objects reuse validation; in-place source mutations trigger revalidation. Invalid replacement documents throw rather than silently serving the previous document.
Each evaluation captures its own snapshot so concurrent calls cannot mix documents. The cached snapshot must not be exposed through returned structured values that callers or validators could mutate. Source data is never mutated.
Time and numeric modulo
Ship all three time Actions in the first implementation using
Intl.DateTimeFormatfor IANA zones and no added timezone runtime dependency. Preserve the validatedCURRENT_TIME,CURRENT_DATETIME, andCURRENT_DAY_OF_WEEKkeys and Python's UTC default. The local store accepts an optionalnow: () => Date; documents do not override the clock.Port conformance cases for inclusive endpoints, same-hour midnight rollover, valid/invalid zones, datetime formats, and daylight-saving boundaries. Correctness must be demonstrated against the current Python baseline rather than inferred from JavaScript Date defaults.
MODULO_RANGEchecks a numeric remainder against inclusiveSTART/ENDbounds using a validatedBASE. It does not hash identifiers. Match Python's modulo behavior, including negative operands, rather than copying JavaScript%blindly. AppConfig traffic splitting is a separate service-owned algorithm; do not promise the same allocation across the two mechanisms.Agent retrieval and Variant normalization
Minimum supported Agent version: 2.0.230173, the version used for the verification below. Verify the corresponding Lambda-extension versions and regional availability before release; observations from a standalone Agent are not proof of native Lambda-extension behavior.
?flag=<name>and normalize the unwrapped flag response. A successful zero-length body means that flag is missing and must be handled before JSON parsing.getAllfetches the whole profile and normalizes the flags it returns.Whole-profile batching is deliberate. In the tested Agent, selective multi-variant requests omit unknown flags, but a basic JSON profile returns 400 for the entire selective request when even one requested flag is absent. Fetching the whole profile preserves both one request and independent Fallbacks.
It can evaluate and transfer unrequested flags through the local HTTP connection; measure that cost for large profiles before adding a more complex recovery strategy.
Normalize
_varianttovariant, preserve the actual AppConfig name, and collect remaining attribute fields without leakingenabled/_variantinto attributes. Missing attributes become{}.A basic flag with no
_variantbecomes a Variant with the synthetic name"default", its actualenabledvalue, and its attributes. Do not replace an actual service-supplied variant name with the synthetic one.disabledVariant()returns{ enabled: false, variant: 'default', attributes: {} }. A caller can supply another valid Variant Fallback explicitly.Agent context encoding
appConfigVariantStoresupports string, finite-number, and boolean context values. Its initial API rejects arrays, nested objects, null/undefined, and nonfinite numbers before retrieval. Array rejection is a supported-API restriction, not a universal claim that every Agent expression is incapable of using arrays.These encoding restrictions apply to remote Agent evaluation only.
appConfigAgentStoreretrieves a freeform document and uses the local rule engine, so it retains the local context contract.Keys must match
^[A-Za-z_][A-Za-z0-9_.-]*$. This is a conservative supported subset. Keys are transmitted without JSON quoting; unsupported keys produce an invalid-context error identifying only the key.JSON-quote string values and escape non-ASCII UTF-16 code units as
\uXXXX; send numbers and booleans as JSON literals. Do not use percent-encoding for context values. Multiple entries can use a combinedContextheader with comma separators because string values are quoted:Context: tier="pro", opted_in=true, limit=10, label="a,b"Verification covered commas, equals signs, embedded quotes, padding, empty strings, Unicode, newlines, and multiple encoded entries. Raw commas/padding changed matching, percent-encoding was not decoded, and quoting keys did not preserve their names.
Header construction must not permit context values to inject another header or entry. Local-engine contexts do not inherit the Agent key restrictions.
Package boundary
Add
@aws-lambda-powertools/feature-flags. The core evaluator and local rule engine use commons and the optional Standard Schema value-validation contract, without bundling a validator implementation. Parameters-backed stores live under/appconfig, with Parameters and JMESPath as optional peers where used. Preserve exact sibling-package pins according to repository standards.The SDK-based factory must not cause Agent-only or custom-store imports to load the AppConfigData SDK at runtime or include it in a tree-shaken bundle. This must be verified for both supported module formats rather than assumed from optional-peer declarations.
Logging
Accept an optional logger. Without one, emit no output and do not fall back to
console. Debug metadata may include the flag name, matched Rule or Flag Default selection, store name, and Variant name. Call-level warnings follow the failure contract above.Never log context values, operand values, raw Agent response bodies, attributes, or Fallback values. Errors may identify flag/rule/key/Action names and safe failure categories; passing a raw error/cause to a logger must not circumvent this policy. No per-evaluation metrics or tracing.
Implementation phases and acceptance criteria
Required verification includes:
Out of scope
validation_exception_handler.Potential challenges
The design decisions are settled for this revision. The following are implementation and release checks, not unspecified fallback behavior:
The decision comment records the real-Agent observations, version/digest, source documentation, and limits of the local experiments. No live IAM denial, AWS service outage, or native Lambda-extension error mapping has been verified yet.
Dependencies and Integrations
@aws-lambda-powertools/commons: core helpers, includingisTruthyandisStrictEqual.@aws-lambda-powertools/parametersas an optional peer for built-in freeform stores. Agent retrieval was introduced in v2.35.0; package manifests pin the current lockstep version rather than using a range.@aws-lambda-powertools/jmespathas an optional peer for envelopes.Alternative solutions
evaluateAllwithout explicit Fallbacks: replaced byevaluateManyso every requested flag has a defined fallback.Acknowledgment
Review status
Future readers
Please react with 👍 and your use case to help us understand customer demand.
All reactions