Data Model Reference

In one sentence. WIP's data model is four building blocks — terminologies/terms (controlled vocabulary), templates (versioned document schemas), documents (validated, identity-addressed instances of a template), and files (binary attachments) — unified by the Registry's canonical IDs and synonym resolution (wip://data-model).

Why it exists. WIP is one generic, template-driven platform, not a bespoke table per app. A template "defines the structure, validation rules, and constraints that documents must conform to" and "support[s] inheritance, allowing child templates to extend parent templates" (template.py, Template docstring). A document "support[s] versioning through identity-based upsert logic," using the identity hash "to detect when a document should be updated (new version) rather than created as a new entity" (document.py, Document docstring). Together, templates + identity give every document class real validation and safe schema evolution (multiple template versions can stay active at once) without every caller inventing its own dedup key.

The model

Terminologies & terms

A terminology is a namespaced controlled vocabulary; a term is one entry in it. Documents store both the value the caller submitted and the resolved term_id (wip://data-model).

(schemas/def-store.json → TerminologyResponse, CreateTerminologyRequest, TerminologyMetadata)

(schemas/def-store.json → TermResponse, CreateTermRequest, TermTranslation)

Terms carry typed ontology relations to each other: is_a, part_of, has_part, regulates, positively_regulates, negatively_regulates, each {source_term_id, target_term_id, relation_type}. Relations support ancestor/descendant/parent/child traversal and bulk import from OBO Graph JSON (wip://data-model; schemas/def-store.json → CreateTermRelationRequest, TraversalResponse).

Templates & fields

(template.py; schemas/template-store.json → TemplateResponse, CreateTemplateRequest)

FieldDefinition — one entry per field:

(field.py; schemas/template-store.json → FieldDefinition, FieldType, ReferenceType, VersionStrategy, SemanticType, FileFieldConfig)

ValidationRule — cross-field rule:

(rule.py; schemas/template-store.json → ValidationRule, RuleType, Condition)

reporting (PostgreSQL sync config attached to a template): sync_enabled (default true), sync_strategy (latest_only upsert | all_versions insert-all), table_name (auto-generated from value if unset), include_metadata, flatten_arrays, max_array_elements (1–100, default 10) (template.py → ReportingConfig). The reporting sync's own service-side models (jobs, alerts, search) are a separate surface — see Reporting & Analytics.

Documents

(document.py; schemas/document-store.json → DocumentResponse, DocumentCreateRequest, DocumentStatus, DocumentMetadata)

Two document-derived projections exist for consumers that don't want the full document shape:

• PeerProjection — a compact view of the entity at the other end of a relationship edge (?include=peers): {document_id, namespace, template_id, template_value, status, data, metadata}, where data/metadata are populated per the peer template's header_fields resolution (see Gotchas). Inactive peers are still resolved and returned, not filtered out (document_store.services.document_service; schemas/document-store.json → PeerProjection).
• TableViewResponse — a flattened, paginated table projection of a template's documents: columns[] ({name, label, type, is_array, is_flattened}), rows[], total_documents vs total_rows (arrays can multiply rows), array_handling ∈ flattened, json, none (schemas/document-store.json → TableViewResponse, TableColumn).

Files

A file is a first-class Registry-identified entity stored in MinIO and referenced by documents through file-typed fields, just like any other reference type (document_store/models/file.py). Fields: file_id (UUID7), namespace, filename, content_type, size_bytes, status ∈ orphan (uploaded, not yet referenced by an active document) | active | inactive, metadata {description, tags[], category, custom}.

Registry, namespaces & synonyms (shape only)

These entities live behind components/registry/src, outside this doc's source; the shapes below come from the wire schema only. Behavior belongs to Identity & the Registry and Namespaces & Tenancy.

(schemas/registry.json → NamespaceResponse, NamespaceCreate)

(schemas/registry.json → EntryDetailResponse, Synonym, SourceInfo)

Every entity in the model above (terminology, term, template, document, file) is Registry-identified: the Registry mints the canonical ID and any registered synonyms resolve to it in O(1) (wip://data-model → "Registry & Synonyms").

How it works

Document write (document_store/services/document_service.py):

1. ValidationService validates data against the template's resolved fields, producing term_references / references / file_references arrays for every term/reference/file-typed field.
2. The template's identity_fields are extracted into identity_values; for a bulk write, identity_values is sent to the Registry, which mints or resolves document_id and returns identity_hash (DocumentService.bulk_create, Stage 2). A dry-run validate (no Registry call) computes the SHA-256 hash locally instead (_result_to_validation_response).
3. Document-store looks for an existing active document at that document_id:
   • none found → insert version 1.
   • found, but data + the three reference arrays are byte-identical to the current version → no-op: returns the existing version, is_new=false, no new version, no event (_data_has_changed).
   • found, data changed, template versioned=true (default) → deactivate the current version (status → inactive) and insert version+1 with the same document_id (_create_new_version).
   • found, data changed, template versioned=false → overwrite the existing row in place: same document_id, same version, fresh data/timestamps, no history kept (_overwrite_in_place).
4. A template with empty identity_fields is append-only: every POST is a new, standalone document (nothing to match against), and PATCH is rejected outright with error code append_only.
5. PATCH applies an RFC 7396 JSON Merge Patch to the current data, re-validates against the same template_version the document was created with (not the latest active version — see Gotchas), rejects any attempt to change an identity_fields value (identity_field_change — create a new document with POST instead), and supports if_match for serialized concurrency (bulk_patch apply loop).

Template write (template_store/services/template_service.py):

1. create_template enforces relationship-template shape when usage="relationship": non-empty source_templates/target_templates, and matching source_ref/target_ref fields of reference_type=document whose own target_templates equal the template-level lists (_validate_relationship_template_shape). For any other usage, source_templates/target_templates must be empty.
2. update_template never edits a template in place — it inserts a new template document with the same template_id and version = (max existing version for this value) + 1. Old versions are not deactivated, so several template versions can be active at once, letting existing documents keep validating against the version they were created with while new documents target the latest (update_template docstring: "Creates a NEW template document with incremented version... allows multiple versions to exist simultaneously"). If nothing changed (_template_has_changed), no new version is written.
3. usage, versioned, source_templates, target_templates are carried over unchanged from the original on every update — they are immutable after the template's first version.
4. Draft mode: create with status="draft" to skip reference validation (enables circular or order-independent creation of interdependent templates). POST /templates/{id}/activate validates the whole reachable set of draft references as one unit and activates them all together — any failure activates none (activate_template).
5. Inheritance (InheritanceService.resolve_template): a template with extends must also set extends_version — there is no "resolve to latest parent" fallback (a stored template without it is a data-integrity error, not a cue to float). Fields merge child-over-parent by name, each tagged inherited/inherited_from; rules concatenate with no dedup; identity_fields resolve from the nearest ancestor (walking child→root) that declares any.
6. Delete: soft-delete sets the version to inactive; hard-delete additionally requires the owning namespace's deletion_mode="full". Either is blocked while any other template still extends this one (delete_template).

Rules & invariants

• Identity & upsert. identity_hash is SHA-256 over the template's identity_fields values. Writing the same identity again versions the document in place (or overwrites, for versioned=false) rather than creating an unrelated new record. Identity is hashed over the raw submitted value (not the resolved term), so identity-bearing term fields must be alias-free (canonical reading list, "identity fields / identity hash").

  ⚠ GAP: the exact normalization performed before hashing (case handling, nested-path extraction, type coercion) is implemented in libs/wip-auth/src/wip_auth/document_identity.py, which is outside this doc's source_scope (document_store/services/identity_service.py is a thin delegate to it and states as much in its module docstring). This doc can confirm the algorithm is SHA-256 over identity_fields values and that it is delegated to that module, not what the normalization function itself does byte-for-byte.

• versioned:false requires identity_fields. An overwrite-in-place template must have an identity to address the row being overwritten; the combination versioned:false + empty identity_fields is rejected at both create and update (_validate_versioned_requires_identity).
• usage is immutable after creation. entity (default, full lifecycle) / reference (reserved) / relationship (typed edge: requires source_templates/target_templates + source_ref/target_ref fields) — set once, never changed.
• versioned is immutable after creation, though identity_fields itself can still be edited later — which is why rule 2 above is re-checked on every update, not just create.
• metadata.* cannot carry structural meaning. identity_fields and full_text_indexed must reference top-level data.<field> names; a metadata.-prefixed entry in either slot is rejected. metadata.* is caller-attached context with no schema commitment (_validate_no_metadata_in_declarative_slots).
• full_text_indexed is string-only and sync-gated. Any other field type is rejected; the flag additionally requires the template's reporting.sync_enabled=true (_validate_full_text_indexed_constraints).
• Nested/array template references pin an exact version. template_ref and array_template_ref both require their matching *_version field — nested data always validates against one fixed (template_id, version) pair, never "latest".
• extends pins extends_version. Inheritance never floats to the parent's latest version.
• Template updates always version forward. No template version is ever mutated; an update is a new document with the same template_id and a higher version. Multiple versions can be simultaneously active.
• Identity fields are immutable via PATCH. Changing an identity_fields value through PATCH is rejected (identity_field_change); the caller must POST a new document.
• A template that other templates extends cannot be deleted (soft or hard) until those children are removed or repointed.
• PoNIF #1 — inactive peers still resolve. The relationships/peer-projection machinery does not filter by document status: an edge whose peer document is inactive still returns that peer rather than erroring or omitting it. See PoNIFs.

Gotchas

• header_fields: [] does not mean "project nothing but identity." An explicitly empty list is indistinguishable from an absent one and falls through to the identity_fields auto-include tier. To project identity fields only, list them explicitly in header_fields — an explicit non-empty list is the only thing that wins outright over the auto-include (document_service.py, peer-projection header-fields resolution, three-tier fallback: explicit header_fields → identity_fields (+ title/doc_status if the template declares them) → ["title", "doc_status"]).
• The title/doc_status auto-include only fires if the template actually declares those fields (inheritance-resolved) — a template without a title field never gets one synthesized into its peer projection.
• A "no-op" write still returns 200 with is_new=false. Bulk-first: every write returns 200 with per-item results: a byte-identical POST comes back as unchanged, not as an error and not as a new version.
• PATCH validates against the document's pinned template_version, not the latest. If that version was later deactivated, the PATCH fails with template_inactive rather than silently upgrading the document onto a newer schema.
• inactive is overloaded. The same DocumentStatus.INACTIVE value is set both automatically (a version superseded by a newer one) and explicitly (a soft-deleted document) — the status alone doesn't tell you which happened; version/is_latest_version and the audit trail do (document_service.py: _create_new_version vs delete_document).
• Append-only templates (identity_fields=[]) reject PATCH outright, with error code append_only — every change to that data is a brand-new document, never an edit.

See also

• Identity & the Registry — the full identity/synonym/merge lifecycle behind the Registry entities sketched above.
• Namespaces & Tenancy — isolation modes, deletion_mode, and namespace lifecycle behind the Namespace shape sketched above.
• PoNIFs — the full catalogue of non-intuitive platform behaviours, including PoNIF #1 above.