diff --git a/.maise/docs/00-overview.md b/.maise/docs/00-overview.md new file mode 100644 index 0000000..2c738c9 --- /dev/null +++ b/.maise/docs/00-overview.md @@ -0,0 +1,170 @@ +# 00 — Executive Overview + +## Purpose + +Serialize.Linq is a software library. An application adds the library to its own +code. The library converts an expression tree to a text format and back. + +An expression tree is a data structure. It represents a query or a filter as +data, not as running code. The host platform's query technology (LINQ, +Language Integrated Query) builds expression trees when a developer writes a +query or a filter in code. + +Serialize.Linq lets a program: + +- Turn an expression tree into JSON text, XML text, or plain text. +- Store that text, or send it to another process. +- Turn the text back into a working expression tree later. +- Compile the restored expression tree into a runnable delegate. + +**Confidence: High.** These facts come from the product README and from the +public entry points in the source code. + +## Business Domain + +The business domain is **data interchange for query and filter logic**. Many +applications build a filter in one process (for example, a web front end) and +must apply that filter in another process (for example, a database server, a +queue consumer, or a stored rule engine). Serialize.Linq exists to move that +filter safely between processes. + +**Confidence: High.** + +## Main Users + +Serialize.Linq has one class of user: a **software developer** who adds the +library to a host application. The developer writes code that calls the +library. The library has no screen, no end-user workflow, and no direct human +user. + +A second, smaller group is the **library maintainer**, who reviews +contributions, fixes reported defects, and publishes new versions. + +See [07-user-roles-and-permissions.md](07-user-roles-and-permissions.md). + +**Confidence: High.** + +## Major Capabilities + +| Capability | Description | Confidence | +|---|---|---| +| Convert expression to entity tree | Turn an expression tree into an intermediate, serializable entity tree. | High | +| Convert entity tree to wire text | Turn the entity tree into JSON, XML, or plain text. | High | +| Convert wire text back to entity tree | Parse JSON, XML, or plain text back into the entity tree. | High | +| Convert entity tree back to expression | Rebuild a working expression tree, and optionally compile it. | High | +| Restrict which types rebuild | Let the host application block dangerous or unexpected types during rebuild. | High | +| Handle very large expressions safely | Reshape long chained conditions so rebuilding never overflows the call stack. | High | +| Work across many host runtimes | Run on old and new versions of the host platform, and on non-Windows systems. | High | + +## System Context + +Serialize.Linq is an **embedded API**, not a network service. A host +application links the library directly into its own process. There is no +client-server relationship between the library and its caller. + +The library reaches outside its own process in one narrow case: when it +rebuilds an expression tree, it must resolve type names against the types +already loaded in the host process. See +[06-apis-and-integrations.md](06-apis-and-integrations.md) and +[08-business-rules.md](08-business-rules.md) for the security rule that +controls this step. + +**Confidence: High.** + +## Architecture Overview + +The library has one internal pipeline with two directions. + +``` + ┌─────────────────────┐ + Expression │ Entity Tree │ Wire Text + Tree ───────▶│ (mirrors the │──────────▶ (JSON, XML, + (input) │ expression shape) │ or plain text) + └─────────────────────┘ + ▲ │ + │ ▼ + Reverse direction: Wire Text → Entity Tree → Expression Tree +``` + +Three internal layers cooperate: + +1. **Entity layer.** Holds one entity type for every kind of expression node, + plus entity types for reflection facts (a referenced type, method, field, + property, or constructor). See [05-data-and-storage.md](05-data-and-storage.md). +2. **Assembly layer.** Walks a live expression tree and produces the matching + entity tree. Also simplifies captured local values into plain constant + entities, and reshapes very long chained conditions into a balanced shape. + See [02-architecture.md](02-architecture.md). +3. **Format layer.** Converts an entity tree to and from JSON text, XML text, + or plain text, and manages the list of extra types the format layer must + know about ahead of time. See [06-apis-and-integrations.md](06-apis-and-integrations.md). + +**Confidence: High.** + +## Major "Backend Services" + +None. Serialize.Linq has no backend service and no microservice. It is a +library that runs inside the host application's own process. + +**Confidence: High.** No network listener, no service host, and no +independently deployable process exist anywhere in the source tree. + +## Major Frontend Applications + +None. Serialize.Linq has no frontend and no micro-frontend. It has no user +interface of any kind. See [04-ui-specification.md](04-ui-specification.md) +for the full statement of this finding. + +**Confidence: High.** + +## Infrastructure Overview + +Serialize.Linq needs no database, cache, message queue, search engine, or +object store to run. It keeps no state between calls other than settings the +host application sets on its own objects. + +The library's **build and release pipeline** does depend on infrastructure +external to the library itself: + +| Infrastructure | Role | Mandatory? | +|---|---|---| +| Source control host | Stores source code, runs the build pipeline. | Mandatory for building/releasing the library. | +| Continuous integration runner (Windows-based) | Builds, tests, and packages the library. | Mandatory for releasing the library. | +| Package registry | Distributes the built package to host applications. | Mandatory for distribution. | + +See [05-data-and-storage.md](05-data-and-storage.md) and +[10-operational-requirements.md](10-operational-requirements.md). + +**Confidence: High.** + +## External Systems + +| External system | Role | Confidence | +|---|---|---| +| Package registry | Where the built library is published. Host applications pull the library from here. | High | +| Source control host | Hosts the source code and the release pipeline. | High | + +No other external system exists. Serialize.Linq makes no network call, reads +no configuration file, and calls no external service at run time. + +**Confidence: High.** + +## Risks + +| Risk | Description | Confidence | +|---|---|---| +| Unrestricted type rebuild | If a host application rebuilds an expression tree from an untrusted source without setting a type restriction, an attacker-supplied payload can name any type already loaded in the host process. | High — the library ships a specific security control for this and a specific exception type for the rejection case. | +| Silent skip on release pipeline | The release pipeline skips a duplicate package version instead of failing the build. A missed version bump means no new package reaches users, with no build failure to flag it. | High — confirmed in the pipeline configuration. | +| Legacy restore artifacts | The source tree still contains an old package-restore tool and configuration file that the current build no longer uses. | Medium — the files appear unused, but no direct proof of removal safety was found. | + +## Open Questions + +See [13-open-questions.md](13-open-questions.md) for the full list. Key items: + +- No production deployment topology exists to review, because the product is + a library. Any "deployment" question really means "how does a host + application configure and call the library," which this specification + answers only from the library's own public surface. +- The real-world adoption pattern (which kinds of host applications use + Serialize.Linq, and for what business purpose) is not visible from the + source code and is marked as unknown. diff --git a/.maise/docs/01-functional-specification.md b/.maise/docs/01-functional-specification.md new file mode 100644 index 0000000..c4459ca --- /dev/null +++ b/.maise/docs/01-functional-specification.md @@ -0,0 +1,201 @@ +# 01 — Functional Specification + +## Product Scope + +Serialize.Linq converts an expression tree to text, and text back to an +expression tree. It does one job, and it does that job in three text formats. +It has no other product scope. + +**Confidence: High.** + +## Functional Scope + +| In scope | Confidence | +|---|---| +| Convert an expression tree to an intermediate entity tree, and back. | High | +| Convert the entity tree to JSON text, XML text, or plain text, and back. | High | +| Simplify captured local values into plain constant entities during conversion. | High | +| Reshape very long chained conditions to avoid a stack overflow during conversion. | High | +| Register extra types the format layer must know about ahead of time, manually or automatically. | High | +| Restrict which types may be rebuilt when the entity tree comes from an untrusted source. | High | +| Allow a caller-supplied plug-in for extra type conversion, custom serialization behavior, or a custom list of loaded assemblies to search when rebuilding a type. | High | + +## Out-of-Scope Functionality + +| Out of scope | Confidence | +|---|---| +| Converting an expression tree to a binary format. | High — removed in version 4.0 because the binary approach used a mechanism with known security weaknesses. | +| Running or evaluating a query against a real data source. | High — the library only converts and rebuilds the expression tree. Running the rebuilt tree against data is the host application's job. | +| Any user interface, report, dashboard, or visual output. | High — see [04-ui-specification.md](04-ui-specification.md). | +| Any network transport (sending the serialized text to another machine). | High — the host application must move the text itself, for example over its own network call or storage layer. | +| Any scheduled or background processing inside the library. | High — every operation runs synchronously, on the calling thread, when the host application calls it. | +| User authentication or user permission management. | High — the library has no concept of a signed-in user. | + +## Major Features + +### Feature: Serialize an expression tree to text + +Convert a query or a filter, held in memory as an expression tree, into JSON +text, XML text, or plain text, so the host application can store it or send +it elsewhere. + +See [09-workflows.md](09-workflows.md) — Serialization Workflow. + +### Feature: Deserialize text back to an expression tree + +Convert JSON text, XML text, or plain text back into a working expression +tree, and optionally compile it into a runnable delegate. + +See [09-workflows.md](09-workflows.md) — Deserialization Workflow. + +### Feature: Restrict rebuildable types + +Let the host application supply an allow-list of types, or a custom rule, so +that rebuilding text from an untrusted source cannot construct an unexpected +or dangerous type. + +See [08-business-rules.md](08-business-rules.md) — Rule: Type Access Control. + +### Feature: Automatic and manual type registration + +Let the host application either list every extra type the format layer needs +to know about ahead of time, or let the library discover those types on its +own by inspecting the constant values inside the expression tree. + +See [08-business-rules.md](08-business-rules.md) — Rules: Known Type +Registration, Automatic Known Type Discovery. + +### Feature: Safe handling of very large expressions + +Automatically reshape a very long chain of "and"/"or" conditions into a +balanced shape before conversion, so that converting, rebuilding, or +compiling the expression never overflows the call stack. + +See [08-business-rules.md](08-business-rules.md) — Rule: Deep Expression +Reshaping. + +### Feature: Extension points for custom behavior + +Let the host application plug in a custom rule for finding loaded assemblies, +a custom conversion rule for values that do not convert automatically, or a +custom serialization behavior for a type the format layer cannot handle on +its own. + +See [06-apis-and-integrations.md](06-apis-and-integrations.md). + +## User Capabilities + +Serialize.Linq has one category of user capability, exercised entirely +through the calling code the developer writes: + +- Convert an expression tree to text (any of the three formats). +- Convert text back to an expression tree. +- Configure conversion settings (which members to include, how relaxed type + names should be, whether to restrict rebuildable types). +- Register or discover extra types needed by the format layer. + +There is no other user-facing capability, because there is no other user +than the developer. See [07-user-roles-and-permissions.md](07-user-roles-and-permissions.md). + +## Administrative Capabilities + +None inside the library itself. The closest equivalent is the **library +maintainer's** release process: bump the version number, and the release +pipeline publishes a new package. See +[10-operational-requirements.md](10-operational-requirements.md). + +**Confidence: High.** + +## Business Capabilities + +The core business capability is **moving filter and query logic across a +process boundary without losing its exact shape and meaning**. This supports +scenarios such as: + +- A client application builds a filter and sends it to a server for + execution. +- An application stores a filter as text and re-applies it later. +- An application passes a filter to a background job or a separate service. + +**Confidence: Medium** — these usage scenarios are inferred from the +product's stated purpose and are not directly observable in the source code. + +## Reporting + +None. The library produces no report of any kind. + +**Confidence: High.** + +## Import/Export + +The whole library is, in effect, an import/export mechanism for expression +trees: + +- **Export**: convert an expression tree to JSON, XML, or plain text. +- **Import**: convert JSON, XML, or plain text back to an expression tree. + +No other import or export format or capability exists. + +**Confidence: High.** + +## Notifications + +None. The library raises no notification, sends no message, and calls no +notification service. + +**Confidence: High.** + +## Scheduled Behavior + +None inside the library. The **release pipeline** runs on a push event to the +main line of source control, not on a schedule. See +[10-operational-requirements.md](10-operational-requirements.md). + +**Confidence: High.** + +## Error Handling + +The library signals a failure by raising an exception. It defines three +specific exception types, each tied to one failure condition: + +| Failure condition | Result | Confidence | +|---|---|---| +| A constant value does not match its declared type during rebuild. | The library raises an exception naming the mismatched type. | High | +| A referenced method, field, property, or constructor cannot be found again on the resolved type during rebuild. | The library raises an exception naming the declaring type and the missing member's signature text. | High | +| A resolved type is rejected by the host application's type restriction rule. | The library raises an exception naming the rejected type. | High | +| The host application asks for text output/input using a format that does not support text (for example, a binary-only format). | The library raises a general invalid-operation failure. | High | +| An unrecognized expression kind reaches the assembly layer. | The library raises a general argument failure naming the unrecognized kind. | High | +| Reading or writing the underlying text stream fails. | The library wraps the failure in a general serialization failure. | High | + +See [08-business-rules.md](08-business-rules.md) for the full rule list. + +## Validation Behavior + +- A constant entity's value is checked against its declared type before it is + accepted; a mismatch is rejected. (Rule: Constant Type Validation.) +- A resolved type is checked against the host application's type + restriction rule, if one is set, every time the type is resolved — even if + the type name was already resolved once before. (Rule: Type Access + Control.) +- No other structural validation exists; the library assumes the entity tree + it receives correctly mirrors a real expression tree. + +**Confidence: High.** + +## Data Ownership + +Serialize.Linq owns no persistent data. Every entity tree, and every wire-format +text value, exists only for the duration of one conversion call, inside the +calling application's own memory. The host application owns all data before, +during, and after calling the library. + +**Confidence: High.** + +## Business Constraints + +| Constraint | Description | Confidence | +|---|---|---| +| No binary serialization. | Removed in version 4.0 for security reasons; must not be reintroduced. | High | +| Restricted deserialization must be opt-in and explicit. | The host application must construct and pass a restriction rule; without one, all types resolve as before, to preserve compatibility with existing callers. | High | +| Supported platform-version range must stay wide. | The library targets nine different platform versions, including two old Windows-only host-platform versions and two portable baseline profiles, to stay usable in old and new host applications alike. | High | +| New bug fixes need a reproducing test. | The project's own contribution guidance requires this. | High | diff --git a/.maise/docs/02-architecture.md b/.maise/docs/02-architecture.md new file mode 100644 index 0000000..baab1c8 --- /dev/null +++ b/.maise/docs/02-architecture.md @@ -0,0 +1,224 @@ +# 02 — Architecture + +## System Context + +Serialize.Linq is a single embedded **API** component. A host application +loads the library into its own process and calls it directly, in the same +way it calls any other library code. There is no network hop between the +host application and Serialize.Linq. + +``` +┌───────────────────────────────┐ +│ Host Application │ +│ (any host-platform application)│ +│ │ +│ ┌─────────────────────────┐ │ +│ │ Serialize.Linq API │ │ +│ │ (embedded component) │ │ +│ └─────────────────────────┘ │ +└───────────────────────────────┘ + │ ▲ + │ resolves type names │ (reads, at rebuild time only) + ▼ │ +┌───────────────────────────────┐ +│ Types already loaded in the │ +│ host process │ +└───────────────────────────────┘ +``` + +**Confidence: High.** + +## Component Overview + +| Component | Category | Description | +|---|---|---| +| Expression Serialization API | API (embedded, in-process) | The only runtime component. Converts an expression tree to and from JSON, XML, or plain text. | +| Continuous Integration Pipeline | (build/release automation, not a runtime component) | Builds, tests, packages, and publishes the library. See [10-operational-requirements.md](10-operational-requirements.md). | + +No other component exists. See "Components Not Present" below. + +**Confidence: High.** + +## Backend Services + +None. See [00-overview.md](00-overview.md) — "Major Backend Services." + +## Microservices + +None. + +## Frontends + +None. See [04-ui-specification.md](04-ui-specification.md). + +## Micro-Frontends + +None. + +## APIs + +One API surface, entirely in-process (not reachable over a network): + +- **Conversion operations** — turn an expression tree into an entity tree, + and the reverse. +- **Text serialization operations** — turn an entity tree into JSON, XML, or + plain text, and the reverse. +- **Convenience operations** — single-call shortcuts that combine conversion + and text serialization (for example, "give me the JSON text for this + expression tree" in one call). + +Full operation list: [06-apis-and-integrations.md](06-apis-and-integrations.md). + +**Confidence: High.** + +## Data Flow + +### Serialization direction + +1. The host application hands the library a live expression tree. +2. If the tree contains a very long chain of "and"/"or" conditions, the + library reshapes it into a balanced shape first. (Rule: Deep Expression + Reshaping.) +3. The library walks the tree and builds a matching entity tree. While doing + so, it replaces references to captured local values (from the host + application's own closures) with plain constant entities, so the entity + tree does not depend on the host application's internal, non-serializable + objects. +4. The library hands the entity tree to the chosen format layer (JSON, XML, + or plain text), which walks the entity tree and produces text. +5. The text is returned to the host application. Serialize.Linq keeps no copy. + +### Deserialization direction + +1. The host application hands the library a text value (and, optionally, a + rebuild context carrying a type restriction rule). +2. The format layer parses the text back into an entity tree. +3. The library walks the entity tree and rebuilds a live expression tree. + Every time it must resolve a type by name, it checks the type restriction + rule, if one was supplied. A rejected type stops the rebuild with an + exception. (Rule: Type Access Control.) +4. The rebuilt expression tree is returned to the host application, which + may compile and run it. + +**Confidence: High.** Traced directly from the conversion and format-layer +components. + +## Authentication + +Serialize.Linq performs no authentication of its own. It has no concept of a +signed-in user, a session, or a credential. + +The library's **release pipeline** authenticates itself to the package +registry using a short-lived token obtained from the source control host's +identity mechanism, instead of a long-lived stored secret. See +[10-operational-requirements.md](10-operational-requirements.md). + +**Confidence: High.** + +## Authorization + +Serialize.Linq has one authorization-like control: the **type restriction +rule**, applied only during rebuild (deserialization). The host application +supplies an allow-list of permitted types, or a custom rule function. The +library checks every type it resolves against this rule and rejects a type +that fails the check, even if that type name was already resolved earlier in +the same rebuild. + +Without an explicit rule, every type resolves without restriction. This +default favors compatibility with existing callers over safety, so the host +application must opt in to the restriction. + +See [07-user-roles-and-permissions.md](07-user-roles-and-permissions.md) and +[08-business-rules.md](08-business-rules.md). + +**Confidence: High.** + +## Deployment Topology + +Serialize.Linq has no deployment topology of its own. It is packaged as a +distributable library unit and becomes part of whatever host application +includes it. The host application's own deployment topology governs where +and how the combined result runs. + +**Confidence: High.** + +## Runtime Environments + +The library builds for nine target platform versions, covering both the +older Windows-only platform lineage and the newer, cross-platform lineage, +plus two portable baseline profiles. This lets one release run inside host +applications built on very different platform versions, from long-lived +Windows-only systems to the newest cross-platform runtime. + +**Confidence: High.** + +## Scaling + +Serialize.Linq holds no shared state between calls, other than settings +objects the host application creates and owns itself. Each conversion call is +independent. Scaling the library means scaling the host application; the +library places no additional constraint on that. + +**Confidence: High.** + +## High Availability + +Not applicable. The library has no running instance, no uptime, and no +health check. Availability is entirely a property of the host application. + +**Confidence: High.** + +## Disaster Recovery Assumptions + +Not applicable to the library's runtime behavior, because it holds no data. +The only disaster-recovery-relevant asset is the source code and release +history, held by the source control host and the package registry. + +**Confidence: High.** + +## Security Boundaries + +The one meaningful security boundary in the whole system is drawn **during +rebuild (deserialization) of untrusted text**: + +``` + Untrusted text ──▶ Format layer ──▶ Entity tree ──▶ Rebuild step + │ + ▼ + Type restriction rule (optional, + host-supplied) checks every + resolved type name + │ + ┌───────────────┴───────────────┐ + ▼ ▼ + Type allowed → rebuild continues Type rejected → exception, + rebuild stops +``` + +Without a type restriction rule, text from an untrusted source can cause the +rebuild step to construct **any type already loaded in the host process**, +because the default type-resolution behavior searches every loaded assembly +by name. This is the library's documented equivalent of a well-known, older +security weakness in the host platform's own, now-removed binary +serialization mechanism (see [01-functional-specification.md](01-functional-specification.md) — +"Out-of-Scope Functionality"). + +See [08-business-rules.md](08-business-rules.md) — Rule: Type Access Control, +and [13-open-questions.md](13-open-questions.md). + +**Confidence: High.** + +## Components Not Present + +The following component categories, listed in the documentation template, +were checked for and were not found in Serialize.Linq: + +| Category | Present? | Why not | +|---|---|---| +| Backend Service / Microservice | No | No network listener or hosted process exists. | +| Frontend / Micro-Frontend | No | No user interface exists. | +| Data Store, Cache, Search Engine, Object Storage, Message Queue | No | The library holds no persistent or shared runtime state. | +| Identity Provider | No | The library authenticates no user. | +| Scheduler, Background Worker, Event Processor | No | Every operation is synchronous and caller-driven; nothing runs on a timer or a queue. | + +**Confidence: High.** diff --git a/.maise/docs/03-services-and-frontends.md b/.maise/docs/03-services-and-frontends.md new file mode 100644 index 0000000..d24fb12 --- /dev/null +++ b/.maise/docs/03-services-and-frontends.md @@ -0,0 +1,73 @@ +# 03 — Backend Services and Frontends + +## Summary + +Serialize.Linq has **no backend service, no microservice, no frontend, and no +micro-frontend**. It has exactly one runtime component: an embedded API that +runs inside the host application's own process. This document records that +finding against every field the documentation template requires, so the gap +is explicit rather than silent. + +**Confidence: High.** + +## Component: Expression Serialization API + +| Field | Value | +|---|---| +| Name | Expression Serialization API | +| Category | API (embedded, in-process — not network-reachable) | +| Purpose | Convert an expression tree to JSON, XML, or plain text, and convert that text back into an expression tree. | +| Why it exists | To let a host application move query or filter logic across a process boundary — over a network call, into storage, or between components — without losing its exact structure. | +| Responsibilities | Walk an expression tree and build a matching entity tree; reshape overly deep condition chains; simplify captured local values into constants; convert the entity tree to and from text; manage the list of extra types the format layer must know ahead of time; rebuild an expression tree from an entity tree, applying an optional type restriction rule. | +| Inputs | An expression tree (for serialization); JSON, XML, or plain text, plus optional settings and an optional rebuild context (for deserialization). | +| Outputs | JSON, XML, or plain text (for serialization); a rebuilt expression tree, optionally compiled into a runnable delegate (for deserialization). | +| Dependencies | The host process's own loaded types, discovered through the platform's built-in reflection facility. No external service dependency. | +| Data owned | None persisted. All data is transient, scoped to one call, and owned by the caller before and after the call. | +| External interactions | Reads the list of types already loaded in the host process when resolving a type name during rebuild. No outbound network call, no file access, no external service call. | +| APIs exposed | See [06-apis-and-integrations.md](06-apis-and-integrations.md) for the full operation list. | +| APIs consumed | None. | +| Background processing | None. Every operation is synchronous and runs on the calling thread. | +| Security responsibilities | Enforce the host-supplied type restriction rule during rebuild, for every resolved type, on every resolution (including repeat resolutions of an already-seen type name). | +| Failure behavior | Raises a typed exception for each of the specific failure conditions in [01-functional-specification.md](01-functional-specification.md) — "Error Handling." Does not retry, does not degrade gracefully, and does not log. The failure always surfaces directly to the calling code. | +| Scaling considerations | None beyond the host application's own scaling. The component holds no shared or cross-call state. | + +**Confidence: High.** + +## Component: Continuous Integration Pipeline + +This is a build/release automation component, not a runtime component of the +delivered library. It is documented here for completeness because it is the +only other identifiable "component" in the repository. + +| Field | Value | +|---|---| +| Name | Continuous Integration Pipeline | +| Category | Not a listed runtime category — a build and release automation process. | +| Purpose | Build the library for every supported target, run the automated test suite, package the library, and publish the package to the package registry. | +| Why it exists | To make every release repeatable, tested, and traceable, without a manual, error-prone release step. | +| Responsibilities | Restore dependencies using a locked, reproducible dependency list; build; run tests; package; publish to the package registry only from the main line of source control. | +| Inputs | Source code changes pushed to, or proposed against, the main line of source control. | +| Outputs | A built and tested package, uploaded as a build artifact; on a successful push to the main line, a published package on the package registry. | +| Dependencies | The source control host's build-runner service; the package registry's publishing endpoint; a short-lived authentication token obtained through the source control host's identity mechanism. | +| Data owned | None business-relevant. Owns only its own build artifacts and logs. | +| External interactions | Publishes to the package registry; authenticates through the source control host's identity mechanism. | +| APIs exposed | None. | +| APIs consumed | The package registry's publish operation. | +| Background processing | Runs on each qualifying push or proposed change; not on a fixed schedule. | +| Security responsibilities | Uses a short-lived, narrowly scoped token instead of a stored long-lived credential, to reduce the impact of a leaked credential. | +| Failure behavior | A duplicate version publish is silently skipped rather than treated as an error; every other step fails the pipeline outright on error. | +| Scaling considerations | Not applicable; one pipeline run per triggering change. | + +See [10-operational-requirements.md](10-operational-requirements.md) for +full detail. + +**Confidence: High.** + +## Cross-References + +- Runtime architecture: [02-architecture.md](02-architecture.md) +- Full API operation list: [06-apis-and-integrations.md](06-apis-and-integrations.md) +- Business rules enforced by this component: [08-business-rules.md](08-business-rules.md) +- Workflows this component participates in: [09-workflows.md](09-workflows.md) +- Operational detail for the pipeline: [10-operational-requirements.md](10-operational-requirements.md) +- Rebuild plan entry for this component: [12-reproduction-plan.md](12-reproduction-plan.md) diff --git a/.maise/docs/04-ui-specification.md b/.maise/docs/04-ui-specification.md new file mode 100644 index 0000000..6e2851d --- /dev/null +++ b/.maise/docs/04-ui-specification.md @@ -0,0 +1,94 @@ +# 04 — UI Specification + +## Finding + +Serialize.Linq has **no user interface**. It has no screen, no page, no +dialog, no wizard, no popup, no drawer, and no view of any kind. It is a +code library with no visual output. + +**Confidence: High.** No markup, styling, layout, or presentation code exists +anywhere in the source tree. The library's only outputs are JSON text, XML +text, or plain text, produced for the calling program to consume, store, or +transmit — never for a person to read on a screen. + +This document records that finding against every subsection the +documentation template requires, so the absence of a user interface is +explicit rather than silent. + +## General + +| Field | Value | +|---|---| +| Purpose | Not applicable — no screen exists. | +| Intended users | Not applicable. | +| Entry points | Not applicable. | +| Exit points | Not applicable. | +| Navigation | Not applicable. | +| Permissions | Not applicable. | +| Related workflows | The library's own workflows have no visual step. See [09-workflows.md](09-workflows.md). | + +## Layout + +Not applicable. No header, footer, sidebar, toolbar, main content area, +status bar, panel, tab, or section exists. + +## Components + +Not applicable. No table, list, form, card, chart, tree, calendar, map, +dashboard, editor, file upload, notification, search box, filter, pagination +control, dialog, or wizard exists. + +## Forms + +Not applicable. No field, validation rule, default value, save action, or +cancel action exists in any visual sense. The closest equivalent — the +settings a developer sets in code before calling the library — is documented +as configuration, not as a form, in +[06-apis-and-integrations.md](06-apis-and-integrations.md). + +## Tables + +Not applicable. + +## Screen States + +Not applicable. No initial, loading, empty, success, error, permission +denied, or offline screen state exists. The library's only "state" is +whether a call succeeded or raised an exception, documented in +[01-functional-specification.md](01-functional-specification.md) — +"Error Handling." + +## Responsive Behavior + +Not applicable. No desktop, tablet, or mobile presentation exists. + +## Accessibility + +Not applicable. No keyboard navigation, focus order, label, screen reader +behavior, contrast requirement, or accessible error presentation exists, +because no visual surface exists to make accessible. + +## User Journey + +Not applicable in the visual sense. The closest equivalent is the +**developer's integration journey**: + +1. **Primary flow**: the developer adds the library, writes code to convert + an expression tree to text, and writes code to convert text back. +2. **Alternative flow**: the developer additionally configures a type + restriction rule, custom type conversion, or a custom serialization + behavior. +3. **Error flow**: a call raises one of the exceptions listed in + [01-functional-specification.md](01-functional-specification.md), and the + developer's own calling code decides how to handle it. +4. **Cancellation flow**: not applicable — every operation is a single, + synchronous call with no cancellable, long-running step. + +**Confidence: High.** + +## Recommendation + +Any future team rebuilding Serialize.Linq should not budget time for a user +interface. If a future product wraps Serialize.Linq in a visual tool (for +example, a filter-builder screen for end users), that tool is a separate +product and needs its own, separate specification. diff --git a/.maise/docs/05-data-and-storage.md b/.maise/docs/05-data-and-storage.md new file mode 100644 index 0000000..e996870 --- /dev/null +++ b/.maise/docs/05-data-and-storage.md @@ -0,0 +1,180 @@ +# 05 — Data and Storage + +## Summary + +Serialize.Linq holds **no persistent data**. Its entire data model is a set +of transient business entities that exist only for the duration of one +conversion call, entirely in the calling application's own memory. This +document describes that transient entity model, because it is the core +"business data" of the product, even though none of it is stored. + +**Confidence: High.** + +## Business Entities + +The product's business entity is the **Expression Entity** family: a set of +entity types that together mirror every kind of node a query or filter +expression tree can contain. Each entity type converts one way from a real +expression node, and converts back the other way into a real expression +node. + +### Expression Entities (mirror one kind of expression node each) + +| Entity | Represents | Confidence | +|---|---|---| +| Binary Operation Entity | A two-sided operation, such as a comparison or an arithmetic operation. | High | +| Conditional Entity | An if/then/else choice between two values. | High | +| Constant Value Entity | A literal or captured fixed value. | High | +| Default Value Entity | "The default value of this type." | High | +| Indexer Access Entity | Access through an index, such as an item lookup by position or key. | High | +| Invocation Entity | Calling a stored function value with arguments. | High | +| Lambda (Function Definition) Entity | A parameterized function definition — the outermost shape of most filters. | High | +| List Initializer Entity | Building a collection and adding items to it in one step. | High | +| Member Access Entity | Reading a field or a property from a value. | High | +| Member Initializer Entity | Constructing an object and then setting some of its members. | High | +| Method Call Entity | Calling a named operation on a value or a type. | High | +| New Array Entity | Constructing an array, either with a fixed size or from listed values. | High | +| New Object Entity | Constructing a new instance of a type. | High | +| Parameter Reference Entity | A reference to one of the function definition's own input values. | High | +| Type Test Entity | Testing whether a value is, or is exactly, a given type. | High | +| Unary Operation Entity | A one-sided operation, such as negation or a type conversion. | High | + +### Reference Entities (describe a reflection fact, not an operation) + +| Entity | Represents | Confidence | +|---|---|---| +| Type Reference Entity | The identity of a type, by name, including its generic arguments. | High | +| Method Reference Entity | The identity of a method, resolved again later by matching its full signature text. | High | +| Constructor Reference Entity | The identity of a constructor, resolved the same way. | High | +| Field Reference Entity | The identity of a field, resolved the same way. | High | +| Property Reference Entity | The identity of a property, resolved the same way. | High | +| General Member Reference Entity | The identity of a member that does not fit the four categories above (used, for example, inside a member-initializer entity). | High | + +### Supporting Entities + +| Entity | Represents | Confidence | +|---|---|---| +| Element Initializer Entity | One "add this item" step inside a list-initializer entity. | High | +| Member Binding Entity (three kinds: direct assignment, list-add, nested) | One "set this member" step inside a member-initializer entity. | High | +| Entity List | An ordered group of expression entities, such as a call's argument list. | High | +| Reference Entity List | An ordered group of reference entities. | High | + +**Confidence: High.** This inventory is a direct, complete listing of the +entity types found in the source tree's entity layer. + +## Entity Relationships + +Every expression entity may hold other expression entities as children (for +example, a binary-operation entity holds a left entity and a right entity). +This produces a tree shape that mirrors the original expression tree +exactly, one entity per node. A reference entity never holds an expression +entity as a child; it only carries identifying facts (a name, a declaring +type reference, and, for a method, its signature text and generic +arguments). + +**Confidence: High.** + +## Ownership + +The calling host application owns every entity instance, for the lifetime of +one conversion call. Serialize.Linq creates entities on demand and discards +its own references to them once the call returns. No entity survives past +the call that created it, inside the library itself. + +**Confidence: High.** + +## Lifecycle + +1. **Created** — during serialization, when the assembly layer walks a real + expression tree. +2. **Converted to text** — by the chosen format layer. +3. **Discarded** — the library keeps nothing after returning the text. +4. **Re-created from text** — during deserialization, by the same format + layer, in reverse. +5. **Converted back to a real expression** — by asking each entity to + rebuild its matching expression node, using a shared rebuild context so + that repeated references (for example, to the same function parameter) + resolve to the exact same rebuilt object. +6. **Discarded** — again, the library keeps nothing after returning the + rebuilt expression tree. + +**Confidence: High.** + +## Persistence + +None. Serialize.Linq never writes an entity, or its text form, to disk, to a +database, or to any other durable store on its own. Any persistence is the +host application's own choice and own responsibility, performed entirely +outside the library. + +**Confidence: High.** + +## Retention + +Not applicable — there is nothing to retain. + +## Search + +Not applicable — no search capability exists over expression entities. + +## Cache + +Two small, in-memory lookup caches exist, but only for the duration of one +rebuild context object (not across separate calls, and not shared across +threads doing unrelated work): + +- A cache of resolved types, keyed by type name, so the same type name is not + resolved twice in one rebuild. +- A cache of rebuilt function-parameter objects, keyed by parameter name and + type, so every reference to "the same" parameter inside one rebuild + resolves to the exact same object, which real expression trees require for + correctness. + +Both caches are created fresh for each rebuild context and discarded with it. + +**Confidence: High.** + +## Files + +None. Serialize.Linq reads no file and writes no file. + +## Objects (Blob/Object Storage) + +None. See "Infrastructure" below. + +## Data Synchronization + +Not applicable — there is no second copy of any data to synchronize. + +## Data Migration Assumptions + +None apply to the library's runtime behavior. The one migration-shaped +concern in the product is **wire-format stability**: the entity types are +described in the source as "the serialization contract," meaning a change to +an entity's shape, or to the short internal names used in the size-optimized +build variant, is a breaking change for any text already produced by an +older version. See [08-business-rules.md](08-business-rules.md) and +[11-non-functional-requirements.md](11-non-functional-requirements.md). + +**Confidence: High.** + +## Infrastructure — Data Stores + +None mandatory or optional at run time. Serialize.Linq needs no relational +database, no document database, no cache server, no message queue, no +search engine, and no object storage service to operate. + +| Storage technology | Purpose | Owner | Consumers | Backup | Scaling | Availability | Deployment options | +|---|---|---|---|---|---|---|---| +| None | Not applicable | Not applicable | Not applicable | Not applicable | Not applicable | Not applicable | Not applicable | + +**Confidence: High.** + +## Cross-References + +- Entities are produced and consumed by the Serialization and Deserialization + workflows: [09-workflows.md](09-workflows.md). +- Entities are governed by the business rules in + [08-business-rules.md](08-business-rules.md). +- The single component that owns entity handling is documented in + [03-services-and-frontends.md](03-services-and-frontends.md). diff --git a/.maise/docs/06-apis-and-integrations.md b/.maise/docs/06-apis-and-integrations.md new file mode 100644 index 0000000..9e9699e --- /dev/null +++ b/.maise/docs/06-apis-and-integrations.md @@ -0,0 +1,104 @@ +# 06 — APIs and Integrations + +## Summary + +Serialize.Linq exposes **one API**: a set of in-process operations a host +application calls directly. The API is not reachable over a network, has no +authentication of its own, and has no rate limit, because every call happens +inside the caller's own process on the caller's own thread. + +**Confidence: High.** + +## API: Expression Serialization API + +| Field | Value | +|---|---| +| Purpose | Convert an expression tree to text, and text back to an expression tree. | +| Provider | Serialize.Linq, embedded in the host application. | +| Consumer | The host application's own code. | +| Authentication | Not applicable — in-process call, no identity boundary. | +| Authorization | The optional type restriction rule, applied only on the deserialize (text-to-expression) direction. See [08-business-rules.md](08-business-rules.md) — Rule: Type Access Control. | +| Internal or external | Internal (in-process). | +| Versioning | Governed by the package version number; a breaking change to an entity's shape is a breaking change to previously produced text. | +| Rate limits | Not applicable. | +| Retry behavior | Not applicable — the library performs no retry; a failure is a single exception raised to the caller. | +| Timeouts | Not applicable — every operation is synchronous and unbounded by the library itself. | +| Error handling | See [01-functional-specification.md](01-functional-specification.md) — "Error Handling." | + +### Operations + +| Operation | Direction | Input | Output | +|---|---|---|---| +| Convert expression to entity tree | Serialize (step 1) | An expression tree, plus optional conversion settings. | An entity tree. | +| Convert expression directly to JSON text | Serialize (combined) | An expression tree, plus optional settings or a custom assembly layer / format layer. | JSON text. | +| Convert expression directly to XML text | Serialize (combined) | Same as above. | XML text. | +| Convert entity tree, or expression, to text through a chosen format layer | Serialize (combined, format-agnostic) | An expression tree (or entity tree), plus the chosen format layer. | Text, in whichever format the chosen format layer produces. | +| Serialize to a stream | Serialize | An expression-derived entity tree and an output stream. | The entity tree's JSON, XML, or other stream-based format written to the stream. | +| Serialize to text | Serialize | An expression-derived entity tree. | A JSON or XML string (only for a format layer that supports text). | +| Deserialize from a stream | Deserialize | An input stream, plus optional rebuild context. | A rebuilt expression tree. | +| Deserialize from text | Deserialize | JSON or XML text, plus optional rebuild context. | A rebuilt expression tree. | +| Configure known-type registration | Configuration | One or more types, or a toggle to auto-register array/list variants, or a toggle to auto-discover types by inspecting constant values. | Updated registration state on the serializer. | +| Register a custom value-conversion rule | Configuration | A target type (or "any type") and a conversion function. | Updated conversion state on the internal value converter. | +| Register a custom assembly-loader | Configuration | An implementation supplying the list of assemblies to search when resolving a type name. | Used automatically during every subsequent rebuild through that context. | +| Register a custom serialization behavior for otherwise-unsupported types | Configuration (XML format only) | An implementation of the host platform's serialization-surrogate contract. | Used automatically by the XML format layer; not honored by the JSON format layer, due to a limitation in the underlying JSON engine. | + +**Confidence: High.** Enumerated directly from the library's public entry +points. + +### Binary Format — Removed + +An extension point for a binary format still exists in the API shape (a +contract for byte-array input/output), but **no built-in binary format +implementation ships with the library**. A previous built-in binary +implementation was removed in version 4.0 because its underlying mechanism +carried a well-known security weakness. A host application that wants a +binary format must supply its own implementation of the extension point. + +**Confidence: High.** + +## External Integration: Package Registry + +| Field | Value | +|---|---| +| Purpose | Distribute the built library package to host applications. | +| Data exchanged | The built package (compiled library plus a matching symbols package) and its version-numbered metadata. | +| Trigger | A push of a new commit to the main line of source control. | +| Frequency | On demand, whenever a maintainer merges a change and bumps the version number. | +| Failure behavior | Publishing an already-published version number is silently skipped, not treated as an error. Any other publish failure fails the release pipeline. | + +**Confidence: High.** + +## External Integration: Source Control Host's Identity Mechanism + +| Field | Value | +|---|---| +| Purpose | Let the release pipeline prove its identity to the package registry without a stored, long-lived secret. | +| Data exchanged | A short-lived identity token, exchanged for a short-lived package-registry credential. | +| Trigger | Every release pipeline run that reaches the publish step. | +| Frequency | Once per qualifying pipeline run. | +| Failure behavior | If the token exchange fails, the publish step fails and no package is published. | + +**Confidence: High.** + +## Internal Extension Points (Not External Integrations, but Integration-Shaped) + +These let a host application customize the library's behavior without +modifying the library itself. They are documented here because they are the +library's designed "seams" for integration with a larger system: + +| Extension point | Purpose | +|---|---| +| Custom assembly-loader | Control which loaded assemblies the library searches when resolving a type name during rebuild — for example, to narrow the search in a security-sensitive host. | +| Custom type restriction rule | Restrict which resolved types are acceptable during rebuild. See [08-business-rules.md](08-business-rules.md). | +| Custom value-conversion rule | Teach the library how to convert a value to a type it cannot convert automatically. | +| Custom assembly layer (node factory) | Change how the library decides which parts of an expression tree become entities versus which parts are simplified into plain constants. | +| Custom serialization behavior (XML only) | Let a type that the format layer cannot serialize on its own participate anyway, through the host platform's serialization-surrogate mechanism. | + +**Confidence: High.** + +## Cross-References + +- Business rules enforced through these operations: [08-business-rules.md](08-business-rules.md) +- Workflows built from these operations: [09-workflows.md](09-workflows.md) +- The component providing this API: [03-services-and-frontends.md](03-services-and-frontends.md) +- Release pipeline detail: [10-operational-requirements.md](10-operational-requirements.md) diff --git a/.maise/docs/07-user-roles-and-permissions.md b/.maise/docs/07-user-roles-and-permissions.md new file mode 100644 index 0000000..b71a022 --- /dev/null +++ b/.maise/docs/07-user-roles-and-permissions.md @@ -0,0 +1,106 @@ +# 07 — User Roles and Permissions + +## Summary + +Serialize.Linq has no sign-in, no session, and no traditional permission +system. It has two roles, neither of which is an "end user" in the usual +sense, plus one security control that behaves like an authorization rule. + +**Confidence: High.** + +## User Roles + +| Role | Description | Confidence | +|---|---|---| +| Integrating Developer | Writes the host application's code that calls Serialize.Linq. Chooses the format, sets conversion and known-type settings, and — if handling untrusted input — configures the type restriction rule. This is the library's only true "user." | High | +| Library Maintainer | Reviews contributions, fixes reported defects (each with a reproducing test, per project policy), decides the version number, and triggers a release by merging to the main line of source control. | High | + +No other role exists. There is no administrator role, no read-only role, and +no end-user role, because the library has no user interface and no +multi-tenant runtime state to separate between users. + +**Confidence: High.** + +## Permissions + +Serialize.Linq defines no permission model in the access-control sense. The +nearest equivalent is a set of **settings the Integrating Developer chooses +in code**, each of which changes what the library is allowed to do on the +developer's own behalf: + +| Setting | Effect | +|---|---| +| Allow private member access | Lets the library read and rebuild non-public fields and properties, not only public ones. | +| Relaxed type names | Chooses a shorter, more portable type-name form in the produced text, except for compiler-generated types, which always use the fully qualified form. | +| Known-type registration | Declares which extra types the format layer must be ready to handle. | +| Type restriction rule | Declares which types are allowed to be reconstructed when rebuilding from text. | + +**Confidence: High.** + +## Authentication + +Not applicable at the library level — see +[02-architecture.md](02-architecture.md) — "Authentication." The release +pipeline authenticates itself to the package registry using a short-lived +token; that is a build-time concern, not a runtime permission concern for +the library's own API. + +## Authorization + +The library's one authorization-like control is the **type restriction +rule**, checked during rebuild (deserialization): + +- If the Integrating Developer supplies a rule, every resolved type is + checked against it, and a rejected type stops the rebuild with an + exception naming the rejected type. +- If no rule is supplied, every type resolves without restriction, for + compatibility with code written before this control existed. + +This is the library's documented way of preventing an untrusted payload from +causing the rebuild step to construct an unexpected or dangerous type — the +same problem a different, now-removed serialization mechanism in the +platform is known to have had. + +See [08-business-rules.md](08-business-rules.md) — Rule: Type Access +Control. + +**Confidence: High.** + +## Administrative Users + +Not applicable to the runtime library. At the project level, the Library +Maintainer plays this role for the source code and release process only. See +[10-operational-requirements.md](10-operational-requirements.md). + +## Read-Only Users + +Not applicable. The library has no multi-user state to read. + +## Privileged Operations + +The only "privileged" runtime operation is rebuilding an expression tree +from text obtained from a source the Integrating Developer does not fully +trust. The project's documentation explicitly recommends that this operation +always be paired with a type restriction rule. + +**Confidence: High.** + +## Visibility Rules + +Not applicable — there is no multi-user data to hide or reveal. + +## Security Assumptions + +| Assumption | Confidence | +|---|---| +| The Integrating Developer, not the library, decides whether input text is trusted or untrusted. | High | +| The library will not, on its own, add a type restriction rule; the default is unrestricted, for backward compatibility. | High | +| A type restriction rule, once attached to a rebuild context, cannot be bypassed by resolving the same type name twice — the check re-runs even on a cache hit. | High | +| The library never evaluates or enumerates a constant value's contents while discovering extra types automatically, specifically to avoid triggering a side effect (for example, materializing a lazily-evaluated data source) as an unintended consequence of type discovery. | High | + +## Cross-References + +- Security boundary detail: [02-architecture.md](02-architecture.md) — "Security Boundaries" +- Full rule text: [08-business-rules.md](08-business-rules.md) +- Restricted-deserialize workflow: [09-workflows.md](09-workflows.md) +- Open question about real-world trust boundaries: [13-open-questions.md](13-open-questions.md) diff --git a/.maise/docs/08-business-rules.md b/.maise/docs/08-business-rules.md new file mode 100644 index 0000000..7e4f505 --- /dev/null +++ b/.maise/docs/08-business-rules.md @@ -0,0 +1,344 @@ +# 08 — Business Rules + +## About This Document + +Each rule below was extracted either from the library's core conversion +logic, or from a regression test that guards a specific, previously reported +defect. A rule tied to a regression test carries **High confidence**, +because the test proves the exact triggering scenario and the exact expected +outcome. + +## Rule: Constant Type Validation + +- **Description**: A constant-value entity's stored value must match its + declared type. +- **Trigger**: Setting the declared type on a constant-value entity. +- **Preconditions**: A value is already attached to the entity. +- **Decision logic**: If the value is not an instance of the newly assigned + type (or a compatible type), reject the assignment. +- **Outcome**: The library raises an exception naming the mismatched type. +- **Affected users**: Integrating Developer. +- **Affected services**: Expression Serialization API. +- **Business rationale**: Prevents a corrupted or tampered entity tree from + silently producing a wrong value on rebuild. +- **Confidence**: High. + +## Rule: Type Access Control + +- **Description**: When rebuilding an expression tree from text, every type + resolved by name is checked against an optional, host-supplied allow-list + or custom rule. +- **Trigger**: Resolving a type name to a real type, at any point during + rebuild — including each generic argument of a generic type, checked + individually. +- **Preconditions**: The host application supplied a type restriction rule + on the rebuild context. Without one, this rule does not apply, and all + types resolve as before. +- **Decision logic**: If a rule is present and it rejects the resolved type, + stop the rebuild immediately, even if that exact type name was already + resolved once earlier in the same rebuild (the check is not skipped on a + cache hit). +- **Outcome**: The library raises an exception naming the rejected type, and + the rebuild does not complete. +- **Affected users**: Integrating Developer. +- **Affected services**: Expression Serialization API. +- **Business rationale**: Rebuilding an expression tree from text + reconstructs types by name. Without a restriction, text from an untrusted + source could name any type already loaded in the host process. This rule + closes that gap, mirroring a well-known security lesson from an older, + now-removed serialization mechanism in the platform. +- **Confidence**: High. + +## Rule: Known Type Registration + +- **Description**: The text-format layer must be told, ahead of time, about + any non-obvious concrete type (for example, a custom enumeration) that may + appear as a constant value inside an expression tree, or it cannot + reconstruct that value correctly. +- **Trigger**: Serializing or deserializing an expression tree containing a + constant of such a type. +- **Preconditions**: The type is not already one of the format layer's + built-in known types (basic numbers, text, date/time, and similar simple + types). +- **Decision logic**: If the type was registered (directly, or as an + array/list variant of a registered type), the value serializes and + deserializes correctly. If not, and automatic discovery (see next rule) is + off, the operation fails. +- **Outcome**: A registered type round-trips correctly; an unregistered type + fails serialization. +- **Affected users**: Integrating Developer. +- **Affected services**: Expression Serialization API. +- **Business rationale**: The underlying text-format engines require every + possible concrete type in a polymorphic value slot to be known in advance; + this rule documents that constraint and the developer's manual escape + hatch for it. +- **Confidence**: High. + +## Rule: Automatic Known Type Discovery + +- **Description**: Instead of manually registering every custom type used as + a constant, the developer can turn on automatic discovery, and the library + finds those types itself by inspecting the constant values already present + in the entity tree. +- **Trigger**: Serializing an entity tree, with automatic discovery turned + on. +- **Preconditions**: Automatic discovery is enabled on the serializer. +- **Decision logic**: Walk the entity tree, record the runtime type of every + constant value found, decompose array element types and generic type + arguments, skip anything already known, and register the remainder — all + without evaluating or enumerating the value itself, to avoid an unintended + side effect (for example, running a lazily-evaluated query). +- **Outcome**: Custom types used as constants serialize correctly without a + manual registration step. +- **Affected users**: Integrating Developer. +- **Affected services**: Expression Serialization API. +- **Business rationale**: Reduces a historically manual, error-prone, + easy-to-forget setup step. +- **Confidence**: High. + +## Rule: Automatic Array/List Type Registration + +- **Description**: When a type is registered as known, the developer can + also ask the library to automatically register the array form, or the list + form, of that type at the same time. +- **Trigger**: Registering a known type, with the array or list auto-register + toggle turned on (the two toggles are mutually exclusive). +- **Preconditions**: The expression contains a containment check (for + example, "is this value in this list") against a collection of the + registered type. +- **Decision logic**: Expand the registered type into its array or list form + before adding it to the known-type set. +- **Outcome**: A containment check against an array or a list of a custom or + simple type serializes correctly, including when the list holds a + nullable element type. +- **Affected users**: Integrating Developer. +- **Affected services**: Expression Serialization API. +- **Business rationale**: Containment checks against a collection are common + in real filters; this rule removes a repetitive manual registration step + for that common case. +- **Confidence**: High. + +## Rule: Deep Expression Reshaping + +- **Description**: A very long chain of "and"/"or" conditions is + automatically reshaped into a balanced shape before any conversion begins. +- **Trigger**: Converting an expression tree that contains more than a small + number of chained "and"/"or" terms (observed regression case: fifteen + thousand terms). +- **Preconditions**: None — this check runs on every serialization. +- **Decision logic**: If the chain is long enough, flatten it into a plain + list of terms, then rebuild it as a balanced tree, preserving the original + evaluation order and meaning. +- **Outcome**: Converting, rebuilding, or later compiling the expression + never overflows the call stack, regardless of how many chained terms the + original filter had. +- **Affected users**: Integrating Developer. +- **Affected services**: Expression Serialization API. +- **Business rationale**: A naturally built, deeply chained condition (for + example, a large "value is one of these one thousand IDs" filter expressed + as repeated "or" checks) can be as deep as it has terms; walking such a + tree without reshaping it can crash the process. +- **Confidence**: High. + +## Rule: Captured Value Simplification + +- **Description**: A reference to a value captured from the surrounding code + (a local variable, or a property of a captured object) is simplified into + a plain constant-value entity, rather than serialized as a reference into + the surrounding code's own, non-serializable state. +- **Trigger**: Converting an expression whose body refers to a value from + outside its own declared parameters — including a value captured inside a + guarded block, or inside a paused asynchronous method. +- **Preconditions**: The referenced value is not itself one of the + expression's own declared input types. +- **Decision logic**: If the reference can be evaluated to a concrete value + without side effects, replace it with a constant-value entity holding that + value. +- **Outcome**: The filter serializes correctly and its meaning is preserved, + even though the surrounding code's own internal state is never itself + serialized. +- **Affected users**: Integrating Developer. +- **Affected services**: Expression Serialization API. +- **Business rationale**: This is what lets a developer write an ordinary, + natural filter that references a local variable, without needing to + design that filter around the library's own limitations. +- **Confidence**: High. + +## Rule: Private-Member Read Timing + +- **Description**: When private-member access is turned on, a captured + object's own (instance) field value is captured at serialization time, but + a type's shared (static) field value is re-read at rebuild-and-run time, + not at serialization time. +- **Trigger**: Serializing an expression that reads a non-public field or + property, with private-member access turned on. +- **Preconditions**: Private-member access is enabled; the referenced member + is not publicly accessible. +- **Decision logic**: An instance member's current value is fixed into the + entity tree as a constant at the moment of serialization. A static + member's value is looked up again, live, when the rebuilt expression is + later compiled and run. +- **Outcome**: Two different, deliberate points in time at which a private + value is "read," depending on whether it belongs to an instance or to the + type itself. +- **Affected users**: Integrating Developer. +- **Affected services**: Expression Serialization API. +- **Business rationale**: An instance the developer is filtering on may not + exist anymore by the time the filter is rebuilt and run elsewhere, so its + values must be captured now; a type's shared value is always available + again later, so re-reading it live keeps the filter current. +- **Confidence**: High. + +## Rule: Compiler-Generated Type Resolution + +- **Description**: A type the host compiler generates automatically (for + example, to carry the extra named values introduced by a query's "let" + step) must resolve correctly on rebuild, whether it is declared in the same + compiled unit as the caller, or in an entirely separate one. +- **Trigger**: Converting or rebuilding an expression that contains such a + compiler-generated type. +- **Preconditions**: The expression was built using a language feature that + causes the compiler to generate a supporting type behind the scenes. +- **Decision logic**: Always record such a type using its fully qualified + identity, never the shorter, relaxed form, regardless of the developer's + relaxed-type-name setting. +- **Outcome**: The expression round-trips correctly, even across a compiled- + unit boundary. +- **Affected users**: Integrating Developer. +- **Affected services**: Expression Serialization API. +- **Business rationale**: Compiler-generated types are easy to miss when + designing type-resolution logic, because a developer never names them + directly; this rule guards a defect class specific to that blind spot. +- **Confidence**: High. + +## Rule: Anonymous and Dynamically-Typed Result Support + +- **Description**: An expression that constructs an anonymous, on-the-fly + object shape, or that is declared to return a dynamically-typed result, + must still serialize and deserialize correctly. +- **Trigger**: Converting such an expression. +- **Preconditions**: None beyond the expression shape itself. +- **Decision logic**: Treat the anonymous shape like any other constructed + object shape; resolve the dynamic return type using the same + fully-qualified-name handling as other compiler-generated types. +- **Outcome**: The expression round-trips correctly. +- **Affected users**: Integrating Developer. +- **Affected services**: Expression Serialization API. +- **Business rationale**: Projecting query results into a lightweight, + purpose-built shape is a very common query pattern. +- **Confidence**: High. + +## Rule: Indexer Access Support + +- **Description**: Access through an index (for example, a dictionary lookup + by key) must be representable as its own kind of entity, not silently + dropped or misrepresented. +- **Trigger**: Converting an expression containing an indexer access, + including one introduced by rewriting a strongly-typed member access into + a dictionary-style lookup. +- **Outcome**: The expression serializes, deserializes, and — once rebuilt + and compiled — behaves the same as the original. +- **Affected users**: Integrating Developer. +- **Affected services**: Expression Serialization API. +- **Business rationale**: Supports flexible, dictionary-backed filter models + built by rewriting a typed filter into a loosely-typed one. +- **Confidence**: High. + +## Rule: Default-Value Expression Support + +- **Description**: "The default value of this type" must be representable as + its own kind of entity. +- **Trigger**: Converting an expression containing such a value. +- **Outcome**: The expression round-trips to an equal expression. +- **Affected users**: Integrating Developer. +- **Affected services**: Expression Serialization API. +- **Business rationale**: A previously unsupported expression shape that + real filters can produce. +- **Confidence**: High. + +## Rule: Self-Referential Type Safety + +- **Description**: Discovering the members and known types reachable from a + type must not loop forever, even when that type refers back to itself + (directly, through a shared field of its own type, or through an ordinary + instance field). +- **Trigger**: Serializing a call to a method on an object whose type + contains such a self-reference, with the type registered as known. +- **Decision logic**: Track types already visited while walking the type + graph, and stop revisiting them. +- **Outcome**: Serialization completes without an unbounded loop or a + related failure. +- **Affected users**: Integrating Developer. +- **Affected services**: Expression Serialization API. +- **Business rationale**: Real object models frequently contain + self-referential or mutually-referential shapes; the library must not + assume a strictly tree-shaped type graph. +- **Confidence**: High. + +## Rule: Custom Serialization Behavior (XML Only) + +- **Description**: A developer may plug in a custom serialization behavior + for a type the format layer cannot handle on its own, but only for the XML + format. +- **Trigger**: Serializing or deserializing through the XML format layer, + with a custom serialization behavior configured. +- **Decision logic**: The XML format layer passes the custom behavior + through to the underlying XML engine. The JSON format layer does not, + because of a limitation in its underlying engine. +- **Outcome**: A type otherwise unsupported by the format layer can still + participate, through XML only. +- **Affected users**: Integrating Developer. +- **Affected services**: Expression Serialization API. +- **Business rationale**: Gives the developer an escape hatch for edge-case + types, while being explicit about the one-format limitation so the + developer does not rely on it accidentally through JSON. +- **Confidence**: High. + +## Rule: Parameter Identity Preservation + +- **Description**: Every reference, inside one rebuilt function definition, + to "the same" input parameter must resolve to the exact same rebuilt + object, not to separate, look-alike objects. +- **Trigger**: Rebuilding a function-definition entity with more than one + reference to the same parameter. +- **Decision logic**: Cache the rebuilt parameter object by its name and + type, for the lifetime of one rebuild context, and reuse it for every + matching reference. +- **Outcome**: The rebuilt function definition is structurally valid and can + be compiled and run. +- **Affected users**: Integrating Developer. +- **Affected services**: Expression Serialization API. +- **Business rationale**: The host platform's own expression-tree rules + require this exact-object identity; without it, the rebuilt expression is + invalid. +- **Confidence**: High. + +## Rule: Member Resolution by Signature Text + +- **Description**: A referenced method, field, property, or constructor is + never serialized directly. It is serialized as its declaring type plus its + full signature text, and re-resolved by matching that text again on + rebuild. +- **Trigger**: Every serialization and rebuild involving a member reference. +- **Decision logic**: On rebuild, resolve the declaring type, then scan its + members (respecting the private-access setting) for one whose signature + text matches exactly. +- **Outcome**: If no member matches, the rebuild fails with an exception + naming the declaring type and the expected signature text. +- **Affected users**: Integrating Developer. +- **Affected services**: Expression Serialization API. +- **Business rationale**: Reflection facts cannot be serialized directly + across process or version boundaries; matching by signature text is the + library's chosen, portable substitute. +- **Confidence**: High. + +## Cross-References + +- Rules are exercised by the workflows in [09-workflows.md](09-workflows.md). +- Rules are enforced inside the component documented in + [03-services-and-frontends.md](03-services-and-frontends.md). +- The entities these rules govern are documented in + [05-data-and-storage.md](05-data-and-storage.md). +- The security-relevant rule (Type Access Control) is also discussed in + [02-architecture.md](02-architecture.md) and + [07-user-roles-and-permissions.md](07-user-roles-and-permissions.md). diff --git a/.maise/docs/09-workflows.md b/.maise/docs/09-workflows.md new file mode 100644 index 0000000..daf483d --- /dev/null +++ b/.maise/docs/09-workflows.md @@ -0,0 +1,159 @@ +# 09 — Workflows + +## Workflow: Serialize an Expression Tree + +- **Goal**: Turn a query or filter, held as a live expression tree, into + JSON text, XML text, or plain text. +- **Actors**: Integrating Developer (through the host application's own + code). +- **Preconditions**: The host application already holds a valid expression + tree in memory. +- **Trigger**: The host application calls a serialize operation. +- **Services involved**: Expression Serialization API. +- **Frontends involved**: None. +- **Step-by-step flow**: + 1. The host application passes the expression tree, and optional settings, + to the library. + 2. The library checks for a very long chain of "and"/"or" conditions and + reshapes it into a balanced shape if needed. (Rule: Deep Expression + Reshaping.) + 3. The library walks the expression tree and builds a matching entity + tree, simplifying any captured local value into a plain constant along + the way. (Rule: Captured Value Simplification.) + 4. If automatic type discovery is on, the library inspects the entity + tree's constant values and registers any extra types it finds. (Rule: + Automatic Known Type Discovery.) + 5. The chosen format layer converts the entity tree into text. + 6. The library returns the text to the host application. +- **Data changes**: None persisted. The entity tree and the text both exist + only for this call. +- **Success**: Text is returned, ready to store or transmit. +- **Failure**: An unrecognized expression kind, an unregistered custom type, + or a stream failure raises an exception, and no text is returned. See + [01-functional-specification.md](01-functional-specification.md) — + "Error Handling." +- **Alternative paths**: The developer may call a single combined operation + that performs steps 1 through 6 in one call, instead of converting to an + entity tree first and formatting it second. +- **Recovery behavior**: None automatic. The host application's own calling + code decides whether to retry, fall back, or surface the failure. + +**Confidence: High.** + +## Workflow: Deserialize Text Back to an Expression Tree + +- **Goal**: Turn previously produced JSON, XML, or plain text back into a + working expression tree, ready to compile and run. +- **Actors**: Integrating Developer. +- **Preconditions**: The host application holds text previously produced by + this same product (or a compatible producer of the same wire format). +- **Trigger**: The host application calls a deserialize operation. +- **Services involved**: Expression Serialization API. +- **Frontends involved**: None. +- **Step-by-step flow**: + 1. The host application passes the text, and optionally a rebuild context, + to the library. + 2. The chosen format layer parses the text back into an entity tree. + 3. The library walks the entity tree and asks each entity to rebuild its + matching expression node. + 4. Every time a type must be resolved by name, the library resolves it + against the types already loaded in the host process (or against a + custom-supplied list), and — if a rebuild context carries a type + restriction rule — checks the resolved type against that rule. (Rule: + Type Access Control.) + 5. Every reference to the same function parameter resolves to the same + rebuilt object. (Rule: Parameter Identity Preservation.) + 6. Every referenced method, field, property, or constructor is re-resolved + by matching its signature text against the declaring type's members. + (Rule: Member Resolution by Signature Text.) + 7. The library returns the rebuilt expression tree to the host + application. +- **Data changes**: None persisted. +- **Success**: A working expression tree is returned. The host application + may compile it into a runnable delegate and run it. +- **Failure**: A rejected type, a missing member, or a malformed constant + value raises an exception, and no expression tree is returned. +- **Alternative paths**: None beyond choosing the input format (JSON, XML, + or plain text) and whether to supply a rebuild context. +- **Recovery behavior**: None automatic. + +**Confidence: High.** + +## Workflow: Deserialize Untrusted Text Safely + +This is the same workflow as above, with one additional precondition and one +additional actor responsibility, because it is the security-sensitive case +the project's own documentation calls out explicitly. + +- **Goal**: Rebuild an expression tree from text the host application does + not fully trust, without letting the rebuild construct an unexpected or + dangerous type. +- **Actors**: Integrating Developer. +- **Preconditions**: The Integrating Developer has decided the input text + is untrusted, and has built a type restriction rule listing exactly the + types the expected expressions may legitimately use. +- **Trigger**: The host application calls the deserialize operation, passing + a rebuild context that carries the type restriction rule. +- **Services involved**: Expression Serialization API. +- **Step-by-step flow**: Identical to "Deserialize Text Back to an + Expression Tree," with the type-restriction check at step 4 now active for + every resolved type, including every generic argument of a generic type, + checked individually, and re-checked even for a type name already + resolved once earlier in the same rebuild. +- **Data changes**: None persisted. +- **Success**: The expression tree is rebuilt, using only allowed types. +- **Failure**: The first disallowed type encountered stops the rebuild with + an exception naming that type. +- **Alternative paths**: The Integrating Developer may instead supply a + custom rule function rather than an explicit allow-list, for cases the + allow-list shape cannot express cleanly. +- **Recovery behavior**: None automatic. The host application must decide + how to respond to a rejected payload (for example, refuse the request and + log the attempt, in the host application's own logging — Serialize.Linq + itself logs nothing). + +**Confidence: High.** + +## Workflow: Publish a New Library Version + +- **Goal**: Ship a new, tested version of the library to the package + registry. +- **Actors**: Library Maintainer. +- **Preconditions**: A change is ready to merge to the main line of source + control, and the version number has been increased. +- **Trigger**: The Library Maintainer merges (pushes) the change to the main + line of source control. +- **Services involved**: Continuous Integration Pipeline, Package Registry, + Source Control Host's Identity Mechanism. +- **Frontends involved**: None. +- **Step-by-step flow**: + 1. The pipeline restores dependencies using the locked, reproducible + dependency list, failing outright if the lock file is missing or + out of date. + 2. The pipeline builds the library for every supported target. + 3. The pipeline runs the full automated test suite. + 4. The pipeline packages the library and its symbols. + 5. The pipeline uploads the package as a build artifact. + 6. Only if the triggering event was a push to the main line: the pipeline + exchanges a short-lived identity token for a short-lived package- + registry credential, then publishes the package. +- **Data changes**: A new package version becomes available on the package + registry. +- **Success**: The new version is visible on the package registry. +- **Failure**: A failed restore, build, or test step stops the pipeline + before packaging. A publish attempt for an already-published version + number is silently skipped, not treated as a failure. +- **Alternative paths**: A pull request against the main line runs steps 1 + through 5 only, as a check, without publishing. +- **Recovery behavior**: The Library Maintainer must notice a skipped + publish (due to a forgotten version bump) manually; the pipeline gives no + distinct signal for this case. + +**Confidence: High.** + +## Cross-References + +- Rules exercised by these workflows: [08-business-rules.md](08-business-rules.md) +- Services and their responsibilities: [03-services-and-frontends.md](03-services-and-frontends.md) +- Operations called during these workflows: [06-apis-and-integrations.md](06-apis-and-integrations.md) +- Operational detail for the publish workflow: [10-operational-requirements.md](10-operational-requirements.md) diff --git a/.maise/docs/10-operational-requirements.md b/.maise/docs/10-operational-requirements.md new file mode 100644 index 0000000..9efd18e --- /dev/null +++ b/.maise/docs/10-operational-requirements.md @@ -0,0 +1,127 @@ +# 10 — Operational Requirements + +## Required Services + +None at run time. A host application that includes Serialize.Linq needs no +additional running service to use it. See +[00-overview.md](00-overview.md) — "Major Backend Services." + +**Confidence: High.** + +## Required Frontends + +None. See [04-ui-specification.md](04-ui-specification.md). + +## Required Infrastructure + +| Infrastructure | When required | Confidence | +|---|---|---| +| None, at run time | Not applicable | High | +| Source control host | To build and release a new library version | High | +| Continuous integration runner, Windows-based | To build and release a new library version — required because two of the nine build targets are older Windows-only platform versions | High | +| Package registry | To distribute a new library version | High | + +**Confidence: High.** + +## Configuration + +The library has no external configuration file, environment variable, or +connection string. Every setting is a value the Integrating Developer sets +directly on the library's own objects in code: + +| Setting group | Examples | +|---|---| +| Conversion settings | Relaxed type names, private-member access. | +| Known-type settings | Manually registered types, automatic array/list expansion, automatic discovery. | +| Rebuild context settings | Private-member access on rebuild, type restriction rule, custom assembly-loader. | + +**Confidence: High.** + +## Scheduling + +No runtime scheduling exists. The release pipeline runs on a push event, not +on a timer. See [09-workflows.md](09-workflows.md) — "Publish a New Library +Version." + +## Logging + +The library performs no logging of its own. Every failure is raised as an +exception; the host application decides whether and how to log it. + +**Confidence: High.** + +## Monitoring + +Not applicable at run time — there is no running instance to monitor. Build +and release health is visible through the continuous integration pipeline's +own run history and uploaded artifacts. + +**Confidence: High.** + +## Alerting + +None built in. Any alerting on a failed release pipeline run is the source +control host's own notification behavior, not a Serialize.Linq feature. + +## Security + +- The library ships a specific security control (the type restriction rule) + for the one identified runtime security risk (unrestricted type + reconstruction during rebuild of untrusted text). See + [08-business-rules.md](08-business-rules.md). +- The built package is signed with a private cryptographic key on the + Windows build runner, and with a public-only (delay) signature on any + non-Windows build host. +- The release pipeline authenticates to the package registry using a + short-lived token obtained through the source control host's identity + mechanism, instead of a stored long-lived credential. + +**Confidence: High.** + +## Backup + +Not applicable to the library's own runtime, which holds no data. The +project's source history and release history are backed up by the source +control host and the package registry, outside the library's own control. + +## Restore + +The library's own build uses a locked, reproducible dependency list, and the +release pipeline fails outright if that list is missing or does not match +the resolved dependency graph. This is a reproducibility control for +building the library, not a data-restore control. + +**Confidence: High.** + +## Scaling + +Not applicable. See [02-architecture.md](02-architecture.md) — "Scaling." + +## Availability + +Not applicable to the library's own runtime. See +[02-architecture.md](02-architecture.md) — "High Availability." + +## Deployment + +The library is "deployed" only in the sense of being published as a package +and pulled into a host application's own build. There is no separate +deployment step, environment, or runtime to stand up for the library itself. + +**Confidence: High.** + +## Runtime Requirements + +The library needs a host process built on one of nine supported target +platform versions: two older Windows-only host-platform versions, five +recent cross-platform host-platform versions, and two portable baseline +profiles. No other runtime prerequisite exists. + +**Confidence: High.** + +## Cross-References + +- Release workflow: [09-workflows.md](09-workflows.md) +- The pipeline component: [03-services-and-frontends.md](03-services-and-frontends.md) +- Security control detail: [08-business-rules.md](08-business-rules.md) +- Rebuild plan for this operational model: [12-reproduction-plan.md](12-reproduction-plan.md) diff --git a/.maise/docs/11-non-functional-requirements.md b/.maise/docs/11-non-functional-requirements.md new file mode 100644 index 0000000..e7c6e72 --- /dev/null +++ b/.maise/docs/11-non-functional-requirements.md @@ -0,0 +1,30 @@ +# 11 — Non-Functional Requirements + +| Quality attribute | Requirement | Status | Confidence | +|---|---|---|---| +| Performance | Converting and rebuilding an expression tree must not overflow the call stack, even for a very long chained condition (observed regression case: fifteen thousand terms), handled by reshaping the chain into a balanced shape before conversion. | Explicit | High | +| Scalability | Not meaningfully applicable — the library holds no shared state and each call is independent; scaling is entirely the host application's concern. | Inferred | High | +| Availability | Not applicable — no running instance exists. | Inferred | High | +| Reliability | Every documented failure condition raises one specific, typed exception rather than failing silently or returning a wrong result; a mismatched constant type and a missing member are explicitly rejected rather than tolerated. | Explicit | High | +| Security | Rebuilding an expression tree from untrusted text can be restricted to an explicit allow-list of types, closing a known class of type-confusion / untrusted-deserialization risk that a previous, now-removed binary format in this same product line was vulnerable to. | Explicit | High | +| Privacy | Not directly addressed. The library serializes whatever constant values the expression tree contains, including any sensitive data the host application chose to capture as a constant; the library provides no data classification or redaction feature. | Inferred | Medium | +| Compliance | No compliance framework, standard, or certification is referenced anywhere in the project. | Unknown | Unknown | +| Auditability | The library performs no logging and keeps no audit trail. Any audit trail is the host application's own responsibility. | Explicit | High | +| Maintainability | New defect fixes are required by project policy to ship with a reproducing automated test; the test suite already covers roughly two dozen previously reported defects, one file per defect, each testing one specific scenario. | Explicit | High | +| Extensibility | Five distinct extension points let a host application customize type discovery, type restriction, value conversion, entity-tree construction, and (for XML only) custom serialization behavior, without modifying the library. | Explicit | High | +| Testability | The project maintains an automated test suite, including a second, separate compiled unit used specifically to test type resolution across a compiled-unit boundary. No numeric test-coverage target is stated in the project's own documentation. | Explicit (suite exists); Unknown (coverage target) | High / Unknown | +| Localization | Not applicable — the library produces no human-facing text, only machine-readable JSON, XML, or plain text. | Inferred | High | +| Internationalization | Not applicable, for the same reason. | Inferred | High | +| Accessibility | Not applicable — no user interface exists. | Inferred | High | +| Disaster recovery | Not applicable to the library's own runtime, which holds no data. The project's source and release history depend on the source control host's and package registry's own disaster-recovery posture, which is outside this specification's visibility. | Unknown | Unknown | +| Business continuity | The release process depends on one named individual's ownership of the package registry publishing identity, as recorded in the release pipeline configuration; no documented succession or backup-maintainer process was found. | Inferred | Medium | +| Operational support | No support channel commitment (response time, coverage hours) is documented; the project accepts community contributions and issue reports on its source-control host. | Inferred | Medium | +| Portability | The library targets nine distinct platform versions, spanning two decades of the host platform's evolution, specifically to remain usable across very old and very new host applications alike. | Explicit | High | +| Backward compatibility | The default behavior for type resolution during rebuild remains fully unrestricted unless a host application explicitly opts into the newer type-restriction control, preserving behavior for callers written before that control existed. | Explicit | High | + +## Cross-References + +- Security requirement detail: [08-business-rules.md](08-business-rules.md), [02-architecture.md](02-architecture.md) +- Extensibility points: [06-apis-and-integrations.md](06-apis-and-integrations.md) +- Test-driven defect history: [08-business-rules.md](08-business-rules.md) +- Open items needing validation: [13-open-questions.md](13-open-questions.md) diff --git a/.maise/docs/12-reproduction-plan.md b/.maise/docs/12-reproduction-plan.md new file mode 100644 index 0000000..b91d1bc --- /dev/null +++ b/.maise/docs/12-reproduction-plan.md @@ -0,0 +1,162 @@ +# 12 — Reproduction Plan + +## Purpose + +This is a blueprint for rebuilding Serialize.Linq from this specification +alone, without access to the original source code. It orders the work into +milestones, states what each milestone must deliver, and states how to know +each milestone is done. + +**Confidence: High** for the component, data-model, and API scope of each +milestone below, since each one is a direct restatement of a High-confidence +finding elsewhere in this specification. **Confidence: Medium** for the +milestone ordering itself, since a rebuilding team could reasonably resequence +some steps (for example, building both text formats together rather than one +per milestone) without changing the outcome. + +## Required Components + +| Component | Required? | Reference | +|---|---|---| +| Expression Serialization API (the only runtime component) | Required | [03-services-and-frontends.md](03-services-and-frontends.md) | +| Continuous Integration Pipeline | Required for release, not for the library's own runtime behavior | [10-operational-requirements.md](10-operational-requirements.md) | +| Backend Service, Frontend, Data Store, Message Queue, or any other infrastructure category | Not required | [02-architecture.md](02-architecture.md) — "Components Not Present" | + +## Required Frontends + +None. Do not budget design or front-end engineering time for this product. +See [04-ui-specification.md](04-ui-specification.md). + +## Required Data Model + +Implement the full Expression Entity family, plus the Reference Entity and +Supporting Entity families, exactly as inventoried in +[05-data-and-storage.md](05-data-and-storage.md). Every entity must convert +one way from a real expression node, and back the other way, given a shared +rebuild context. + +## Required APIs + +Implement every operation listed in +[06-apis-and-integrations.md](06-apis-and-integrations.md), across at least +two text formats (a straightforward format and a more compact, structured +format), plus the five extension points listed there. + +## Required Workflows + +Implement, in this order, all four workflows in +[09-workflows.md](09-workflows.md): Serialize, Deserialize, Deserialize +Untrusted Text Safely, and Publish a New Library Version. + +## Required Security Model + +Implement the Type Access Control rule +([08-business-rules.md](08-business-rules.md)) before shipping any version +intended for use with untrusted input. Default to the unrestricted behavior +only if backward compatibility with an existing caller base is a stated +project goal; otherwise, consider defaulting to restricted, since an +unrestricted default is the specification's one identified security risk. + +## Required Operational Capabilities + +- A reproducible, locked dependency-restore mechanism, enforced by the build + pipeline (fail the build if the lock does not match). +- A release pipeline that builds, tests, packages, and — only from the main + line of source control — publishes, using a short-lived credential rather + than a stored long-lived one. +- A duplicate-version-publish guard that does not fail the pipeline, paired + with a way for the maintainer to notice a skipped publish (this + specification's rebuild should improve on the original by adding this + missing feedback signal — see + [13-open-questions.md](13-open-questions.md)). + +## Suggested Implementation Order + +1. **Milestone 1 — Entity model and pure conversion.** + Build the full Expression Entity and Reference Entity family, and the + pure, in-memory conversion step (real expression tree to entity tree, and + back), with no text format yet. + **Acceptance criteria**: every expression kind in + [05-data-and-storage.md](05-data-and-storage.md) converts to an entity + and back to an equal expression, verified by an automated structural + comparison, not by a superficial equality check. + +2. **Milestone 2 — Text formats.** + Add at least one text format layer (for example, a JSON format), plus the + known-type registration mechanism (manual, automatic array/list + expansion, and automatic discovery). + **Acceptance criteria**: a representative expression, including one using + a custom enumeration type and one using a containment check against a + list, serializes and deserializes correctly through the chosen format. + +3. **Milestone 3 — Captured-value simplification and deep-chain safety.** + Add the logic that simplifies captured local values into constants, and + the logic that reshapes very long chained conditions before conversion. + **Acceptance criteria**: an expression referencing a local variable + serializes correctly; a very long chained condition (test with at least + ten thousand terms) converts, rebuilds, and compiles without a stack + overflow. + +4. **Milestone 4 — Type restriction and security hardening.** + Add the rebuild context, the type restriction extension point, and the + specific exception raised on a rejected type. + **Acceptance criteria**: an allow-listed type round-trips; a + non-allow-listed type, including as a generic argument of an otherwise + allowed generic type, is rejected with the correct exception. + +5. **Milestone 5 — Remaining extension points and second text format.** + Add the custom assembly-loader, custom value-conversion, custom + entity-construction, and (for the structured-markup format only) custom + serialization-behavior extension points; add the second text format. + **Acceptance criteria**: each extension point has at least one automated + test proving the host application's custom logic is actually invoked. + +6. **Milestone 6 — Defect-driven hardening pass.** + Work through every rule in [08-business-rules.md](08-business-rules.md) + that traces to a historical defect (nullable values, date/time kind and + epoch handling, compiler-generated types, anonymous/dynamic results, + indexer access, default-value expressions, self-referential types, + private-member read timing, chained method calls, interface-typed + collections combined with bitwise operators), and add one reproducing + automated test per rule before implementing the fix for that rule. + **Acceptance criteria**: every rule in + [08-business-rules.md](08-business-rules.md) has a passing, named, + reproducing test. + +7. **Milestone 7 — Release pipeline.** + Stand up the build, test, package, and publish pipeline described in + [10-operational-requirements.md](10-operational-requirements.md), + including the locked-dependency restore gate and the short-lived + publishing credential. + **Acceptance criteria**: a version bump on the main line of source + control results in a new package on the target package registry, with no + long-lived credential stored anywhere in the pipeline configuration. + +## Risks + +| Risk | Mitigation | +|---|---| +| Underestimating the breadth of expression kinds to support. | Treat the entity inventory in [05-data-and-storage.md](05-data-and-storage.md) as a fixed checklist, not a starting sketch. | +| Treating the type-restriction control as an afterthought. | Build it in Milestone 4, before any general availability release, not as a later patch. | +| Skipping the defect-driven test set because the underlying platform "obviously" handles these cases. | Every rule in [08-business-rules.md](08-business-rules.md) exists because a real, working system got that case wrong once; assume the same risk applies to a fresh implementation. | +| Wide platform-version support becoming a maintenance burden. | Decide explicitly, before Milestone 1, which of the nine original target platform versions the rebuilt product must actually support, rather than defaulting to all of them. | + +## Assumptions + +- The rebuilding team has access to a platform with an equivalent + expression-tree query technology; without one, Milestone 1 is not + meaningful, since the entire product exists to serialize that technology's + own expression trees. +- The rebuilding team treats "the same wire-format shape as the original" as + optional, not required, unless interoperating with existing text produced + by the original product is an explicit goal. + +## Cross-References + +Every milestone above links back to its full specification: architecture in +[02-architecture.md](02-architecture.md), the data model in +[05-data-and-storage.md](05-data-and-storage.md), the API surface in +[06-apis-and-integrations.md](06-apis-and-integrations.md), the rule set in +[08-business-rules.md](08-business-rules.md), the workflows in +[09-workflows.md](09-workflows.md), and operations in +[10-operational-requirements.md](10-operational-requirements.md). diff --git a/.maise/docs/13-open-questions.md b/.maise/docs/13-open-questions.md new file mode 100644 index 0000000..4b86ef7 --- /dev/null +++ b/.maise/docs/13-open-questions.md @@ -0,0 +1,85 @@ +# 13 — Open Questions + +## Missing Information + +| Question | Why it matters | Suggested validation step | +|---|---|---| +| Which kinds of host applications actually use Serialize.Linq, and for what business purpose? | Shapes priority for future feature work and for this specification's assumed usage scenarios. | Ask the Library Maintainer, and review public issue history on the source control host for real-world usage descriptions. | +| Is there a numeric test-coverage target? | Affects confidence in "Maintainability" and "Testability" in [11-non-functional-requirements.md](11-non-functional-requirements.md). | Run the test suite with a coverage tool and compare against the requesting organization's own standard. | +| Are the legacy package-restore tool and configuration file (found alongside the modern lock-file-based restore setup) still needed by anything? | Left unresolved, they are project clutter; if load-bearing for some overlooked scenario, removing them would break something. | Ask the Library Maintainer; attempt a clean build after removing them, on a branch. | +| Is there a defined succession plan if the current Library Maintainer becomes unavailable? | The release pipeline's publishing identity is tied to one named individual. | Ask the Library Maintainer; check the source control host's listed collaborators/owners. | +| What is the actual adopted meaning of "restricted deserialization" in existing host applications — do any currently rely on the unrestricted default against untrusted input? | The unrestricted default is a real security exposure for any caller who has not explicitly opted into the restriction. | Survey known adopters, or add prominent guidance to the product's own README (a Medium-confidence recommendation, not a finding from the code itself). | + +## Ambiguous Behavior + +| Question | Why it matters | Suggested validation step | +|---|---|---| +| One historical test case (nullable local variable comparison) contains a commented-out assertion describing a related scenario that appears never to have been fully resolved. | May indicate an unresolved edge case in nullable-value handling that was deliberately left disabled rather than fixed. | Ask the Library Maintainer; re-enable the assertion on a branch and observe whether it still fails. | + +## Unknown Workflows + +No workflow beyond serialize, deserialize (trusted and untrusted), and +publish-a-release was found. If additional host-application-side workflows +exist (for example, a specific pattern for passing a filter from a web tier +to a data tier), they live entirely in host application code outside this +repository and are not visible here. + +## Unknown Business Rules + +No business rule beyond those listed in +[08-business-rules.md](08-business-rules.md) was found. Any additional rule +would need to come from a defect report or a feature request not yet +reflected in the current source code or test suite. + +## Unknown Infrastructure + +Whether any adopting organization runs its own private package feed (rather +than the public package registry) for internal distribution is unknown and +not visible from this repository. + +## Unknown Deployment + +Not applicable in the traditional sense — see +[02-architecture.md](02-architecture.md) — "Deployment Topology." Any +"deployment" of Serialize.Linq is really the host application's own +deployment, which this repository does not describe. + +## Unknown Permissions + +Not applicable beyond what is documented in +[07-user-roles-and-permissions.md](07-user-roles-and-permissions.md). No +additional permission concept was found. + +## Unknown Integrations + +Whether any host application pairs Serialize.Linq with a specific transport +(a specific message queue product, a specific web framework, a specific +database) is unknown; the library itself is transport-agnostic and makes no +assumption about this. + +## Unknown Operational Assumptions + +| Question | Why it matters | +|---|---| +| Is the Windows-based build runner a hosted, provider-managed runner, or a self-managed one? | Affects the operational risk and maintenance burden of the release pipeline, and was not fully determinable from the workflow file alone. | +| Is there a defined process for responding to a security report against this library specifically (versus the general code-of-conduct contact address found in the repository)? | A dedicated library, especially one with a documented untrusted-deserialization concern, benefits from a clear, separate security-contact process. | + +## Recommended Next Steps + +1. **Product owner / maintainer review** — confirm the assumed usage + scenarios in [00-overview.md](00-overview.md) and + [01-functional-specification.md](01-functional-specification.md) against + real adopter feedback. +2. **Security review** — specifically validate the default-unrestricted + deserialization behavior against the requesting organization's own risk + tolerance, and confirm whether existing callers already apply a type + restriction rule in practice. +3. **Architecture review** — confirm that no additional runtime component + exists outside the reviewed source tree (for example, a companion tool or + service published from a different repository) before treating this + specification as complete. +4. **Independent redevelopment planning** — use + [12-reproduction-plan.md](12-reproduction-plan.md) as the starting + blueprint, and validate each milestone's acceptance criteria against the + business rules in [08-business-rules.md](08-business-rules.md) before + committing to a rebuild timeline. diff --git a/.maise/docs/README.md b/.maise/docs/README.md new file mode 100644 index 0000000..078c61c --- /dev/null +++ b/.maise/docs/README.md @@ -0,0 +1,191 @@ +# Serialize.Linq — Reverse-Engineered Functional Specification + +## Purpose + +This documentation is a reverse-engineered functional specification for +Serialize.Linq. It was built directly from the product's own source code and +tests, not from any external design document. + +The goal is to let an independent engineering team understand, maintain, or +rebuild Serialize.Linq without needing access to the original source code. + +## Scope + +### Covered + +- Functional behavior and business capability +- Architecture and component structure +- The library's one API surface and its extension points +- Business rules extracted from the source code and from historical defect + tests +- Workflows (serialize, deserialize, restricted deserialize, release) +- The data (entity) model +- Infrastructure needed to build and release the library +- Operational requirements +- Security model +- Non-functional requirements + +### Intentionally Not Covered + +- Source code, in any form +- Programming language names, framework names, or build-tool names +- Internal implementation detail not needed to understand behavior +- Package-manager or dependency-specific detail beyond what operational + understanding requires + +## Analysis Summary + +| Item | Count | Confidence | +|---|---|---| +| Backend services | 0 | High | +| Frontend applications | 0 | High | +| Micro-frontends | 0 | High | +| APIs | 1 (embedded, in-process) | High | +| External integrations | 2 (package registry; source-control identity mechanism) | High | +| Data stores | 0 | High | +| User roles | 2 (Integrating Developer; Library Maintainer) | High | +| Business workflows | 4 | High | +| Business entities | 26 (16 expression entities, 6 reference entities, 4 supporting entities) | High | +| Business rules | 16 | High | +| Infrastructure components (build/release only) | 3 (source control host; continuous integration runner; package registry) | High | + +Serialize.Linq is a single embedded code library with no networked +component. Most template categories built for a multi-service application +(frontends, data stores, message queues, identity providers) do not apply +here, and are recorded as explicitly not applicable, with reasoning, rather +than left blank. + +## Documentation Index + +| File | Purpose | +|---|---| +| [00-overview.md](00-overview.md) | Executive summary | +| [01-functional-specification.md](01-functional-specification.md) | Complete functional specification | +| [02-architecture.md](02-architecture.md) | System architecture | +| [03-services-and-frontends.md](03-services-and-frontends.md) | The one backend component; confirms no frontend exists | +| [04-ui-specification.md](04-ui-specification.md) | Confirms no user interface exists, against every template subsection | +| [05-data-and-storage.md](05-data-and-storage.md) | Data (entity) model and storage | +| [06-apis-and-integrations.md](06-apis-and-integrations.md) | API operations and external integrations | +| [07-user-roles-and-permissions.md](07-user-roles-and-permissions.md) | Roles and the security model | +| [08-business-rules.md](08-business-rules.md) | Extracted business rules | +| [09-workflows.md](09-workflows.md) | Business workflows | +| [10-operational-requirements.md](10-operational-requirements.md) | Operational requirements | +| [11-non-functional-requirements.md](11-non-functional-requirements.md) | Quality attributes | +| [12-reproduction-plan.md](12-reproduction-plan.md) | Blueprint for rebuilding the product | +| [13-open-questions.md](13-open-questions.md) | Assumptions and unresolved items | + +## Architectural Summary + +Serialize.Linq converts a query or filter, held as an expression tree in a +host application's memory, into JSON text, XML text, or plain text — and +converts that text back into a working expression tree. It is not a hosted +service; it is code the host application links directly into its own +process and calls like any other library function. + +- **Major component**: one embedded API (the Expression Serialization API), + covering conversion, formatting, known-type management, and restricted + rebuild. +- **Major data stores**: none. All data is transient and owned by the + calling host application. +- **External systems**: a package registry (distribution) and the source + control host's identity mechanism (used only by the release pipeline, to + authenticate a publish action without a stored long-lived credential). +- **Authentication model**: not applicable at run time; the release pipeline + uses short-lived, token-exchange-based authentication to the package + registry. +- **Deployment model**: the library is published as a versioned package and + pulled into whatever host application includes it; the host application's + own deployment model governs the combined result. + +See [00-overview.md](00-overview.md) for the full narrative and a diagram. + +## Confidence Statement + +Nearly every statement in this documentation carries **High confidence**, +because Serialize.Linq is a small, single-purpose library whose complete +behavior is directly observable in its source code and in its automated test +suite — including roughly two dozen tests that each document one specific, +previously reported defect and its fix. + +A small number of statements carry **Medium confidence**, where the source +code shows the mechanism clearly but the real-world business motivation +behind it is inferred rather than stated outright (for example, which kinds +of host applications actually use the library). No statement in this +documentation carries an unlabeled assumption presented as fact; every +inferred or unknown item is called out explicitly, most fully in +[13-open-questions.md](13-open-questions.md). + +## Assumptions + +- The reader is treated as unfamiliar with the original source code, and + every functional claim is traceable to an observed behavior, not to + external knowledge about the product. +- "The host application" is used throughout as the generic term for any + program that includes Serialize.Linq; no specific host application was + identified or assumed. +- Where the template requested information about a component category this + product does not have (a frontend, a data store, a scheduler, and + similar), this documentation states "not applicable" with a reason, rather + than omitting the section. + +## Known Gaps + +- Real-world adoption context (who uses this library, and why) is not + visible from source code alone. See + [13-open-questions.md](13-open-questions.md). +- No stated numeric test-coverage target was found. +- A small number of legacy build-tooling artifacts exist in the repository + whose current relevance could not be confirmed from the code alone. +- Business-continuity planning around the release pipeline's publishing + identity was not found and is flagged as a gap, not confirmed as either + present or absent in practice. + +## Recommendations + +- **Product owner review** — confirm the assumed business purpose and usage + scenarios in [00-overview.md](00-overview.md). +- **Business stakeholder review** — confirm there is no unseen consumer- + facing product built on top of Serialize.Linq that would need its own, + separate specification. +- **User acceptance review** — not applicable in the traditional sense; + substitute a developer-experience review of the API surface in + [06-apis-and-integrations.md](06-apis-and-integrations.md). +- **Architecture review** — confirm no additional component exists outside + the reviewed repository. +- **Operations review** — validate the release pipeline description in + [10-operational-requirements.md](10-operational-requirements.md) against + its live configuration. +- **Security review** — specifically evaluate the unrestricted-by-default + deserialization behavior described in + [08-business-rules.md](08-business-rules.md) and + [02-architecture.md](02-architecture.md) against the requesting + organization's own risk tolerance. +- **Independent redevelopment planning** — use + [12-reproduction-plan.md](12-reproduction-plan.md) as the starting + blueprint. + +## Document Conventions + +- **Confidence levels**: + - **High** — directly observed in the source code or an automated test. + - **Medium** — the mechanism is observed directly, but the stated business + reason behind it is inferred. + - **Low** — a plausible inference with limited direct support. + - **Unknown** — no evidence was found either way. +- **Terminology** — one approved term is used per concept across every + document (for example, "entity" always means a serializable business + object in the data model; "workflow" always means a named, multi-step + business process). The same term is never reused for two different + concepts. +- **Cross-referencing** — every document links to related documents using + its file name; follow these links rather than expecting one document to + repeat another's full detail. +- **Assumptions and open questions** — every assumption is marked inline + where it appears, and every open question is collected in + [13-open-questions.md](13-open-questions.md). + +## Revision Information + +| Version | Date | Description | +|---|---|---| +| 1.0 | 2026-08-12 | Initial reverse-engineered specification, generated from the source tree at commit `e7197c4` (Serialize.Linq version 4.4.1). |