Concepts — Nistor Docs

Facts and the graph#

Nistor never guesses. Every integration reports facts: machine-observed statements such as an asset exists, a lineage edge connects two assets, a run started or failed, a table was last refreshed, a check passed, a user queried a dataset. Facts have a shared, versioned shape defined by the core. Nistor stores them, builds a graph of assets and lineage from them, and evaluates rules and alerts over that graph.

The layers#

The dependency rule is one-way:

core  <-  tools  <-  plugins  <-  distributions

Core is the kernel: the loader, the registry of contracts, the fact sink and the core schema. It names no tool and no plugin.
Tools integrate one external system each (a cloud service, a BI tool, a data-quality tool, a code host). A tool never imports another tool or a plugin.
Plugins add capabilities on top: server routers, background workers, web shell pages. They reach the core only through entry points and registered contracts.
A distribution is a lock of the packages one deployment runs, and the image built from it.

The only way across a layer is an entry point (a named hook a package declares in its metadata, such as nistor.tools, nistor.routers or nistor.executors) or a contract in the core registry. Installing a package is enough to activate what it declares; there is no central list to edit.

Tools#

A tool has a manifest that says what it contributes: its id, the kinds of fact it produces, its facets (for example an executor that launches jobs), a connection type and its settings. Every tool answers a health probe that never raises. A connection is a configured instance of a tool's connection type, with its secrets kept apart from its configuration. See Writing a tool.

Platform roles#

One runtime image runs every server role:

Role	Does
`backend`	the HTTP API behind the web shell
`scheduler`	event-driven scheduling and execution of actions
`compiler`	compiles the configuration into the graph
`input-logs`	receives and enriches the input log events
`jobs`	hosts the platform jobs

nistor serve <role> starts one; further roles exist (for example mcp), and a plugin may ship roles of its own, which nistor serve starts only while that plugin is active. They share one PostgreSQL database and talk to each other over HTTP and a message bus.

Rule tokens#

A rule's strings may carry tokens that each deployment resolves when it matches the rule. The scheduler resolves them again before it runs an action:

Token	Resolves to
`{ENV}`	the tier (`dev`, `test` or `prod`), which names the customer's data estate (`pj-lake-{ENV}`, release paths, SQL)
`{V1_INGEST_ENV}`	the tier's v1-ingest project suffix, read from `environment_mappings.json`
`{INSTANCE}`	the name of a Nistor deployment, which is the suffix of Nistor's own resources (`nistor-backend-{INSTANCE}`)

How {INSTANCE} resolves:

For the deployment's own tier, or when no environment is given, it is that deployment's NISTOR_INSTANCE. It is the tier when that variable is unset.
For another tier it is that tier.
Where a deployment is named by its tier, {INSTANCE} and {ENV} give the same value.
Platform rules target Nistor's own Cloud Run services with {INSTANCE}. A data-estate selector keeps {ENV}.
The CLI cannot know a remote deployment's instance:
- The trigger checks of nistor rules sync skip an {INSTANCE} id as unresolved.
- nistor alerts sync refuses the token.
Rollout order: release the runtime that resolves {INSTANCE} first, and only then switch the deployment configuration's platform_jobs.json from {ENV} to {INSTANCE}. An older backend leaves the token unresolved, and nistor rules check-platform-targets reports that.

Open and commercial parts#

This project contains the core, the platform services, every tool, the web shell and the clients. Commercial plugins are distributed separately and use the same entry points and contracts as any other plugin.

Web shell#

The shell is a React application. A plugin adds pages, navigation and panels through slots, which the shell renders; see Plugin SDK.