Skip to content

Plugins

Plugins are reusable processing components in OpenSPAD.

A plugin definition describes what a plugin is, which schemas it consumes and produces, where it may appear in a pipeline, what it depends on, and whether it is available for use.

Current implementation status

The Plugins administration area is already backed by a real Neo4j plugin registry.

OpenSPAD can currently:

  • list PluginDefinition nodes from Neo4j
  • create or update plugin definitions
  • delete plugin definitions
  • store version and publication status
  • store input and output schema identifiers
  • store dependencies
  • store pipeline position rules
  • store backend endpoint and template metadata

Database-changing operations require the OpenSPAD Administration key.

PluginDefinition concept

A plugin definition is metadata about a reusable OpenSPAD plugin.

It is intentionally separate from a particular project or pipeline run.

flowchart TB
    PD[Plugin Definition]
    PT[Pipeline Template]
    PP[Project Pipeline]
    RUN[Pipeline Run]

    PD --> PT
    PT --> PP
    PP --> RUN

This allows one versioned plugin definition to be reused in many pipeline templates and projects.

Current plugin model

The current backend validates plugin data with the Plugin Pydantic model.

Field Purpose
plugin_id Stable machine-readable plugin identifier
version Plugin definition version
name Human-readable display name
category Plugin category
description Description of the plugin
template Template metadata
backend_endpoint Backend service endpoint
input_schema Declared input schema
output_schema Declared output schema
required Whether the plugin is required
position_rule Allowed pipeline position
status Draft / Published / Disabled
icon UI icon identifier
color UI color metadata
dependencies Other plugin dependencies

The current default schema is:

openspad.pipeline.shared-schema.v1

for both input and output.

Plugin identity and versioning

A plugin is stored using the composite key:

plugin_id|version

For example:

semantic-enrichment|1.0

This means two versions can exist independently:

semantic-enrichment|1.0
semantic-enrichment|1.1

The backend uses Neo4j MERGE on this key, so saving the same plugin_id and version updates the existing definition rather than creating a duplicate.

Plugin ID validation

The current backend requires:

  • minimum length: 2
  • maximum length: 100
  • lowercase letters, numbers and hyphens
  • first character must be a lowercase letter or number

The validation pattern is conceptually:

^[a-z0-9][a-z0-9-]*$

Examples:

semantic-enrichment
ifc-property-sets
bsdd
composer

Plugin contracts

Each plugin definition contains:

input_schema
output_schema

Conceptually:

flowchart LR
    IN[Input Schema]
    P[Plugin]
    OUT[Output Schema]

    IN --> P --> OUT

For enrichment plugins, both may currently be:

openspad.pipeline.shared-schema.v1

That does not mean the plugin does nothing.

It means the structure remains canonical while the content changes.

flowchart LR
    S1[Shared State]
    IFC[IFC Plugin]
    S2[Shared State + IFC]
    BSDD[bSDD Plugin]
    S3[Shared State + IFC + bSDD]

    S1 --> IFC --> S2 --> BSDD --> S3

Position rules

OpenSPAD currently supports four position rules:

first
flexible
late
last

These values are validated by the backend.

first

The plugin is intended to appear at the beginning of the pipeline.

Typical example:

Semantic Enrichment

flexible

The plugin may appear in a normal intermediate position.

Typical examples may include:

IFC
bSDD
Product Data

late

The plugin is intended for a later processing stage.

last

The plugin is intended to be the final processing stage.

Typical example:

Composer

The plugin registry list is sorted by this rule in the order:

first → flexible → late → last

Required plugins

The boolean field:

required

allows a plugin definition to indicate that the plugin is mandatory.

Conceptually, a Pipeline Template can later distinguish between:

flowchart LR
    R[Required Plugin]
    O[Optional Plugin]
    P[Project Pipeline]

    R --> P
    O -. selectable .-> P

The required property is already stored today. Full enforcement by the Pipeline Template layer belongs to the pipeline architecture.

Dependencies

Each plugin definition contains:

dependencies: list[str]

This allows a plugin to state that another plugin must be present.

Example:

["semantic-enrichment"]

Conceptually:

flowchart LR
    SE[Semantic Enrichment]
    IFC[IFC Plugin]
    BSDD[bSDD Plugin]

    SE --> IFC
    IFC --> BSDD

The registry stores these dependencies today.

Dependency validation across complete pipelines is a separate responsibility of the future Pipeline Template / Pipeline Builder validation layer.

Status model

The backend supports exactly three status values:

Draft
Published
Disabled

Draft

The plugin definition is still being prepared.

Published

The plugin is intended to be available for use.

When a plugin is saved with status Published, the backend also sets:

enabled = true

Disabled

The definition remains stored but is not intended for active use.

For non-published states:

enabled = false

Neo4j persistence

Plugin definitions are persisted as:

(:PluginDefinition)

The registry reads them with:

MATCH (p:PluginDefinition)
RETURN properties(p) AS props

and saves them with a merge operation based on:

key = plugin_id + "|" + version

Conceptually:

flowchart TB
    UI[Plugins Administration UI]
    API[Plugin Registry API]
    NEO[(Neo4j)]

    UI --> API
    API --> NEO
    NEO --> API
    API --> UI

Timestamps

When a plugin definition is first created, OpenSPAD stores:

created_at

Every save updates:

updated_at

Both use UTC ISO timestamps.

Current API

List plugins

GET /api/plugin-registry

Returns:

  • ok
  • common_schema
  • plugins

The plugins are returned in position order and then by name and version.

Save plugin

POST /api/plugin-registry/save

The endpoint creates or updates a plugin definition.

Database-changing requests require the header:

X-OpenSPAD-Ontology-Key

The key is compared against the server-side Administration key file.

If the key is missing or invalid:

HTTP 403

is returned.

Delete plugin

POST /api/plugin-registry/delete

The request identifies the plugin using:

key

The matching PluginDefinition node is removed using DETACH DELETE.

This operation also requires the Administration key.

Security model

Read access to the current registry API does not use the mutation key.

Write operations do.

flowchart LR
    READ[GET Registry]
    WRITE[Save / Delete]
    KEY[Administration Key]
    DB[(Neo4j)]

    READ --> DB
    KEY --> WRITE
    WRITE --> DB

The current key file is server-side and is not part of the repository.

Administration UI

The Plugins administration page is available at:

/administration/plugins/

The UI currently supports plugin registry management and displays fields including:

  • Name
  • Plugin ID
  • Version
  • Category
  • Template
  • Input schema
  • Output schema
  • Position rule
  • Required status
  • Publication status

The page also contains the contextual:

? Help

link back to this documentation.

Example: Semantic Enrichment

A conceptual plugin definition could look like:

plugin_id: semantic-enrichment
version: 1.0
name: Semantic Enrichment
category: Enrichment
input_schema: openspad.pipeline.seed.v1
output_schema: openspad.pipeline.shared-schema.v1
required: true
position_rule: first
status: Published
dependencies: []

Conceptually:

flowchart LR
    SEED[Pipeline Seed]
    SE[Semantic Enrichment]
    STATE[Shared Semantic State]

    SEED --> SE --> STATE

Example: Composer

A conceptual Composer definition could look like:

plugin_id: composer
version: 1.0
name: Composer
input_schema: openspad.pipeline.shared-schema.v1
output_schema: ids.document.v1
required: true
position_rule: last
status: Published

flowchart LR
    STATE[Shared Semantic State]
    C[Composer]
    IDS[IDS Document]

    STATE --> C --> IDS

Current status matrix

Capability Status
Plugin administration page Implemented
List plugin definitions Implemented
Create plugin definition Implemented
Update plugin definition Implemented
Delete plugin definition Implemented
Neo4j persistence Implemented
Plugin version field Implemented
Draft / Published / Disabled Implemented
Input / output schema references Implemented
Required flag Implemented
Position rules Implemented
Dependency metadata Implemented
Backend endpoint metadata Implemented
Administration write key Implemented
Central Schema registry enforcement Not yet implemented
Dependency validation across a full pipeline Not yet implemented
Automatic runtime execution from PluginDefinition Not established by this registry code
Pipeline Template compatibility validation Not yet implemented

PluginDefinition vs executable plugin

A PluginDefinition is registry metadata.

It does not by itself prove that the referenced backend endpoint or executable implementation is running.

flowchart LR
    DEF[PluginDefinition]
    META[Metadata + Contract]
    EXEC[Executable Plugin Service]

    DEF --> META
    DEF -. references .-> EXEC

This distinction is important:

  • the registry describes the plugin
  • application code implements the plugin
  • the pipeline runtime eventually coordinates execution

Target pipeline integration

The intended architecture is:

flowchart TB
    REG[Plugin Registry]
    PT[Pipeline Template]
    PB[Pipeline Builder]
    RUN[Pipeline Run]

    REG --> PT
    PT --> PB
    PB --> RUN

A Pipeline Template should select published Plugin Definitions, apply ordering rules, verify dependencies and check schema compatibility.

The user-facing Pipeline Builder can then create a project-specific pipeline from those approved definitions.

What should be implemented next?

The plugin registry itself is already a solid foundation.

Logical next integration steps are:

flowchart TD
    A[Plugin Registry]
    B[Central Schema Registry]
    C[Pipeline Template Registry]
    D[Dependency Validation]
    E[Schema Compatibility Validation]
    F[Runtime Plugin Resolution]
    G[Pipeline Execution]

    A --> B --> C --> D --> E --> F --> G

Central Schema Registry

Replace arbitrary schema strings with selectable registered schema definitions.

Pipeline Templates

Compose published plugin definitions into reusable administrative templates.

Validation

Check:

  • dependencies
  • required plugins
  • position rules
  • input/output schema compatibility

Runtime resolution

Connect the registry definition to the actual executable plugin implementation.

Troubleshooting

Saving returns HTTP 403

The Administration mutation key is missing or invalid.

Neo4j connection fails

The plugin registry reads Neo4j connection settings from the OpenSPAD Neo4j environment configuration. A Neo4j password must be available there.

Saving an existing plugin creates no second row

That is expected when plugin_id and version are unchanged. The registry merges by the composite key and updates the existing definition.

Why can I type schema identifiers that are not centrally registered?

Because the central Schema registry is not implemented yet. Schema references are currently stored as strings.

Does Published mean the executable service is definitely running?

No. Published is registry status. Runtime health and execution are separate concerns.