diff --git a/README.md b/README.md index 8860750..365b70f 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ The documentation establishes the architectural foundation, integration contract ## 📖 Documentation -**Live documentation:** https://OP-TED.github.io/entity-resolution-docs/ +**Live documentation:** https://meaningfy-ws.github.io/entity-resolution-docs/ ## Contents diff --git a/docs/modules/ROOT/images/curation/admin-user-list.png b/docs/modules/ROOT/images/curation/admin-user-list.png index 0b686cc..5d6131a 100644 Binary files a/docs/modules/ROOT/images/curation/admin-user-list.png and b/docs/modules/ROOT/images/curation/admin-user-list.png differ diff --git a/docs/modules/ROOT/images/curation/comparison-panel.png b/docs/modules/ROOT/images/curation/comparison-panel.png index da3e2ea..e187027 100644 Binary files a/docs/modules/ROOT/images/curation/comparison-panel.png and b/docs/modules/ROOT/images/curation/comparison-panel.png differ diff --git a/docs/modules/ROOT/images/curation/decision-area.png b/docs/modules/ROOT/images/curation/decision-area.png index 6f781fb..34e6ba7 100644 Binary files a/docs/modules/ROOT/images/curation/decision-area.png and b/docs/modules/ROOT/images/curation/decision-area.png differ diff --git a/docs/modules/ROOT/images/curation/decision-list.png b/docs/modules/ROOT/images/curation/decision-list.png index ebfbbec..73f21ef 100644 Binary files a/docs/modules/ROOT/images/curation/decision-list.png and b/docs/modules/ROOT/images/curation/decision-list.png differ diff --git a/docs/modules/ROOT/images/curation/filter-bar.png b/docs/modules/ROOT/images/curation/filter-bar.png index 8e4315b..1c0867f 100644 Binary files a/docs/modules/ROOT/images/curation/filter-bar.png and b/docs/modules/ROOT/images/curation/filter-bar.png differ diff --git a/docs/modules/ROOT/images/curation/full-decisions-page.png b/docs/modules/ROOT/images/curation/full-decisions-page.png index 8acbcc1..294bd65 100644 Binary files a/docs/modules/ROOT/images/curation/full-decisions-page.png and b/docs/modules/ROOT/images/curation/full-decisions-page.png differ diff --git a/docs/modules/ROOT/images/curation/metadata.png b/docs/modules/ROOT/images/curation/metadata.png new file mode 100644 index 0000000..72c93e4 Binary files /dev/null and b/docs/modules/ROOT/images/curation/metadata.png differ diff --git a/docs/modules/ROOT/images/curation/needs-rereview-notification.png b/docs/modules/ROOT/images/curation/needs-rereview-notification.png new file mode 100644 index 0000000..22c65e6 Binary files /dev/null and b/docs/modules/ROOT/images/curation/needs-rereview-notification.png differ diff --git a/docs/modules/ROOT/pages/UserGuide/curation-guide-admin.adoc b/docs/modules/ROOT/pages/UserGuide/curation-guide-admin.adoc index ab08b56..96884fe 100644 --- a/docs/modules/ROOT/pages/UserGuide/curation-guide-admin.adoc +++ b/docs/modules/ROOT/pages/UserGuide/curation-guide-admin.adoc @@ -24,6 +24,9 @@ Two independent settings control whether and how a user can use the application. In short: *Active* is the door to the application; *Verified* is the door to curation work inside it. +NOTE: Neither an inactive nor an unverified user can use the application. + + == The User Table Each row in the table represents one user account and shows: diff --git a/docs/modules/ROOT/pages/UserGuide/curation-guide-decisions.adoc b/docs/modules/ROOT/pages/UserGuide/curation-guide-decisions.adoc index 388aee2..558066b 100644 --- a/docs/modules/ROOT/pages/UserGuide/curation-guide-decisions.adoc +++ b/docs/modules/ROOT/pages/UserGuide/curation-guide-decisions.adoc @@ -42,6 +42,11 @@ Scores are colour-coded so you can judge them at a glance: | The engine is most certain. |=== +Each entry may also display a visual indicator of its review status: + +* *Reviewed* — the decision has already been reviewed by a curator. +* *Needs re-review* — the decision was previously reviewed but has been returned to the queue for re-evaluation after a backend update. + === Filtering and Ordering Bar @@ -67,9 +72,14 @@ By default the list shows all pending decisions; use this filter to focus on the | Restrict by similarity score range. Select a predefined band or set custom min/max values. +| *Review status* +| Filter decisions by their review state. +Options: _Never reviewed_ (decisions that have not yet been curated), _Reviewed_ (decisions that have already been curated), _Needs re-review_ (decisions that were previously reviewed but have been returned to the queue after a backend update). +Leave the filter set to _All Statuses_ to see all decisions regardless of review state. + | *Sort by* | Order the decision list. -Options: _Created At (Newest/Oldest)_, _Updated At (Newest/Oldest)_, _Confidence (Low to High / High to Low)_. +Options: _Created At (Newest/Oldest)_, _Updated At (Newest/Oldest)_, _Confidence (Low to High / High to Low)_, _Cluster size (Smallest)_, _Cluster size (Largest)_. | *Search* | Full-text search across entity mention field data (name, identifiers, and other parsed attributes). @@ -102,6 +112,15 @@ It is also where the curator records their feedback. image::curation/decision-area.png[Decision area with comparison panel and alternative clusters] +A *Why review needed* banner at the top of the Decision Area explains why this specific decision was flagged for manual curation. +The message is tailored to the decision and may describe factors such as: high similarity combined with low confidence, the presence of alternative candidate clusters, a singleton cluster (a cluster containing only a single entity), or differences in specific entity property values. +The banner also shows the cluster size and when the decision was last updated. + +Each entity card in the comparison view — both the current entity mention and each member of the proposed cluster — has an *ⓘ* button in the top-right corner. +Clicking it opens a metadata panel with the provenance details for that entity, including its *Source ID*, *Request ID*, and *Entity Type*. + +image::curation/metadata.png[Entity mention metadata panel showing Source ID, Request ID, and Entity Type,60%] + [#alternative-clusters] === Alternative Clusters @@ -140,7 +159,6 @@ To review a decision: . Decide whether the entity mention and the proposed cluster are the same real-world entity. image::curation/comparison-panel.png[Comparison panel with entity cards and diff highlighting] - The highlight colours show *where each field value appears* — they do not represent edits or a before/after history. There are three cases: [cols="1,1,2",options="header"] @@ -160,6 +178,22 @@ The highlight colours show *where each field value appears* — they do not repr | The field is present in both, but the values differ. |=== +==== Previously Reviewed Decisions + +The Decisions Queue may include decisions that were already reviewed or that have been returned for a second look. +When a decision falls into one of these states, the Decision Area displays a notification banner: + +* *Reviewed* — the decision has already been recorded by a curator. + The banner reads: _"This decision has already been reviewed by a curator."_ +* *Needs re-review* — the decision was previously reviewed but has been returned to the queue after a backend update. + The banner reads: _"A curator reviewed this decision previously, but it was returned for curation after a backend update and needs a re-review."_ + +image::curation/needs-rereview-notification.png[Needs re-review notification banner] + +These notifications are informational — you can still inspect the decision and record updated feedback if needed. + + + === Accepting a Decision Accepting a decision registers your feedback that the proposed cluster is a correct match for this entity mention. diff --git a/docs/modules/ROOT/pages/UserGuide/curation-guide-overview.adoc b/docs/modules/ROOT/pages/UserGuide/curation-guide-overview.adoc index 42d928c..9026762 100644 --- a/docs/modules/ROOT/pages/UserGuide/curation-guide-overview.adoc +++ b/docs/modules/ROOT/pages/UserGuide/curation-guide-overview.adoc @@ -66,6 +66,7 @@ From your point of view as a curator, a review cycle runs as follows: . You open a decision and compare the entity mention against the proposed cluster. . You record your verdict — *Accept* the top candidate, *Select* an alternative cluster, or *Reject* all candidates. . Your feedback is stored and forwarded to the engine, which uses it when re-evaluating future decisions. +. Once the engine acts on your feedback and updates a previously reviewed decision, that decision may re-appear in the queue, annotated as *Needs re-review*, indicating it requires another look. . At any time you can open the Action History to review past decisions and their outcomes. == Application Pages diff --git a/docs/modules/ROOT/pages/api-docs/curation/index.adoc b/docs/modules/ROOT/pages/api-docs/curation/index.adoc index 7df00c4..58a8e12 100644 --- a/docs/modules/ROOT/pages/api-docs/curation/index.adoc +++ b/docs/modules/ROOT/pages/api-docs/curation/index.adoc @@ -295,7 +295,7 @@ Accept multiple decisions in a single request. [.alternativeCanonicalEntitiesApiV1CurationDecisionsDecisionIdAlternativeCanonicalEntitiesGet] -==== GET /api/v1/curation/decisions/\{decision_id}/alternative-canonical-entities +==== GET /api/v1/curation/decisions/{decision_id}/alternative-canonical-entities Get Alternative Canonical Entities @@ -385,7 +385,7 @@ Get alternative canonical entities for a given decision. [.decisionApiV1CurationDecisionsDecisionIdAcceptPost] -==== POST /api/v1/curation/decisions/\{decision_id}/accept +==== POST /api/v1/curation/decisions/{decision_id}/accept Accept Decision @@ -457,7 +457,7 @@ Accept the proposed canonical entity match. [.decisionApiV1CurationDecisionsDecisionIdAssignPost] -==== POST /api/v1/curation/decisions/\{decision_id}/assign +==== POST /api/v1/curation/decisions/{decision_id}/assign Assign Decision @@ -542,7 +542,7 @@ Assign the subject entity mention to a specific cluster. [.decisionApiV1CurationDecisionsDecisionIdRejectPost] -==== POST /api/v1/curation/decisions/\{decision_id}/reject +==== POST /api/v1/curation/decisions/{decision_id}/reject Reject Decision @@ -620,7 +620,7 @@ List Decisions ===== Description -Retrieve cursor-paginated list of decisions with optional filtering. +Retrieve cursor-paginated list of decisions with optional filtering. The UI composes the four review states from the two row primitives (``previous_review_count`` and ``reviewed_since_placement``) and filters via the two orthogonal query parameters. Args: filters: Field-level filter criteria (entity type, confidence, etc.). cursor_params: Cursor-based pagination parameters. user: Authenticated and verified curator. service: Decision curation service (injected). ever_reviewed: Filter on lifetime review existence. reviewed_since_placement: Filter on review since the current placement. reviewed: Deprecated alias of ``reviewed_since_placement``. ===== Parameters @@ -635,6 +635,24 @@ Retrieve cursor-paginated list of decisions with optional filtering. |=== |Name| Description| Required| Default| Pattern +| ever_reviewed +| Filter on whether any curator action has ever been recorded against the decision (previous_review_count > 0). Omit to disable. +| - +| null +| + +| reviewed_since_placement +| Filter on whether a curator action exists since the current placement (created_at after updated_at, else created_at). Omit to disable. Combine ever_reviewed=true with reviewed_since_placement=false to list decisions that need re-visiting after an ERE update. +| - +| null +| + +| reviewed +| Deprecated alias of reviewed_since_placement. Ignored when reviewed_since_placement is provided. +| - +| null +| + | entity_type | Filter by entity type | - @@ -728,7 +746,7 @@ Retrieve cursor-paginated list of decisions with optional filtering. [.proposedCanonicalEntityApiV1CurationDecisionsDecisionIdProposedCanonicalEntityGet] -==== GET /api/v1/curation/decisions/\{decision_id}/proposed-canonical-entity +==== GET /api/v1/curation/decisions/{decision_id}/proposed-canonical-entity Get Proposed Canonical Entity @@ -1041,7 +1059,7 @@ Retrieve registry statistics and curation statistics with optional filtering. [.candidatesApiV1UserActionsActionIdCandidatesGet] -==== GET /api/v1/user-actions/\{action_id}/candidates +==== GET /api/v1/user-actions/{action_id}/candidates Get Candidates @@ -1126,7 +1144,7 @@ Get paginated candidate cluster previews with top entity mentions. [.selectedClusterApiV1UserActionsActionIdSelectedClusterGet] -==== GET /api/v1/user-actions/\{action_id}/selected-cluster +==== GET /api/v1/user-actions/{action_id}/selected-cluster Get Selected Cluster @@ -1243,6 +1261,12 @@ List cursor-paginated user actions with optional filtering. | null | +| decision_id +| Filter by the decision identifier. Returns only user actions recorded against this decision (matched via the entity mention triad). +| - +| null +| + | cursor | Pagination cursor from previous response | - @@ -1422,7 +1446,7 @@ Create a new user (admin only). [.userApiV1UsersUserIdPatch] -==== PATCH /api/v1/users/\{user_id} +==== PATCH /api/v1/users/{user_id} Patch User @@ -1795,6 +1819,13 @@ Cluster preview with top entity mentions for display. | Similarity score between the entity mention and the cluster. | +| cluster_size +| X +| +| `Integer` +| Total number of decisions (entity mentions) assigned to this cluster, sourced from the cluster_sizes projection. +| + | top_entities | X | @@ -2099,6 +2130,8 @@ Allowed ordering options for decision listing. | -created_at | updated_at | -updated_at +| cluster_size +| -cluster_size |=== @@ -2149,6 +2182,20 @@ Decision summary for list display. | Timestamp of the last update to this decision. | date-time +| previous_review_count +| +| +| `Integer` +| Lifetime count of curator actions ever recorded against this decision. Persists across ERE re-integrations. Drives the UI 'previously reviewed' indicator. +| + +| reviewed_since_placement +| +| +| `Boolean` +| True iff a curator action exists whose created_at is after the current placement boundary (updated_at, else created_at). Materialised on the decision row by two writers: the integrator resets it to False on every placement advance; record_review conditionally sets it to True when a curator action lands. With previous_review_count the UI composes the review state: count==0 -> not reviewed; count>0 and not this flag -> needs revisit; this flag -> reviewed and up to date. +| + |=== @@ -2481,11 +2528,39 @@ Statistics about the entity registry. | Total number of distinct canonical entity clusters. | -| average_cluster_size +| cluster_size_average | X | | `BigDecimal` -| Average number of entity mentions per canonical entity cluster. +| Average decisions per cluster. +| + +| cluster_size_median +| X +| +| `BigDecimal` +| Median (p50) cluster size. +| + +| cluster_size_p95 +| X +| +| `Integer` +| 95th-percentile cluster size — surfaces long-tail outliers. +| + +| cluster_size_max +| X +| +| `Integer` +| Largest cluster size. +| + +| cluster_singletons_count +| X +| +| `Integer` +| Number of clusters of size 1. | | resolution_requests