Castor

Connect Castor (Coalesce Catalog) and test it; the Catalog module mirrors its descriptions, tags, glossary terms and owners.

Castor is a data catalog, now sold as Coalesce Catalog. The Castor tool (nistor-tool-castor, tool id castor) is the read-only link between a Castor workspace and Nistor: it holds the connection, the GraphQL client and the mapping from Castor's assets to Nistor's nodes. It never writes to Castor.

The package carries the Alpha classifier and its README states no other status (see the overview).

What needs an Enterprise module#

The open-source tool does not move any Castor data into Nistor. The ingestion that reads Castor and mirrors its descriptions, tags, glossary terms and owners into the catalog tables belongs to the Catalog module (Nistor Enterprise). That module reaches the tool's client and mapping code, and runs the castor_ingest poller and the /api/v1/castor routes.

Open source Catalog module
Create a Castor connection and test it yes
Read Castor's GraphQL API with the tool's client yes
Mirror descriptions, tags, glossary terms and owners into the catalog yes

With only the open-source tool, a Castor connection is something you can test, and nothing more.

Create the connection#

Create a connection of type castor in the web shell (Connections, /admin/connections) or with POST /api/v1/connections:

Field Required Meaning
api_url yes Castor's GraphQL endpoint: https://api.castordoc.com/public/graphql for an EU workspace, https://api.us.castordoc.com/public/graphql for a US workspace.
Secret Required Meaning
api_token yes The whole Castor API token, including its castor:: prefix. The prefix is part of the value.

The test sends one small authenticated GraphQL query (getTables, one row, only the total count) and reads Castor's answer on both layers. It is healthy only when Castor answers with a table count. A rejected token is unhealthy: Castor reports it as an UNAUTHENTICATED error inside an HTTP 200, and the probe reads it. A wrong address, a reply that is not a GraphQL answer, an untrusted certificate, a host that does not resolve and a timeout are unhealthy too. The probe never answers degraded, and it reads at most 64 KiB of the reply.

Nistor authenticates with Authorization: Token <token>, not Bearer.

Where credentials are read today#

Nothing reads a Castor connection except its test. The ingestion in the Catalog module reads environment variables, which you set on the roles that run it:

Variable Default Meaning
CASTOR_API_URL https://api.castordoc.com/public/graphql The GraphQL endpoint.
CASTOR_API_TOKEN none The whole castor::<uuid> token. Ingestion is off while it is empty.
CASTOR_HTTP_TIMEOUT_S 30 Timeout of each request.
CASTOR_PAGE_SIZE 50 Rows per page (nbPerPage).
NISTOR_TOOL_LIMIT_CASTOR 24 Concurrent requests per process.

Keep the token in a secret manager, not in a file you commit. The core Terraform module can create the secret castor-api-token-<env> and give the backend access to it (enable_castor), but the reference root does not set that switch or inject the variables: wire them in your deployment root, with the token as a secret reference and not as a plain value. See Install on GCP.

What the tool reads#

The client queries these lists from Castor, a page at a time (the first page is 0): tables, columns, dashboards, glossary terms, tags, pinned assets, users and teams. The tool's mapping code turns them into catalog documents, ownership edges, and links between tags, terms and assets. It also adds up, for each glossary term, the query counts and popularity that Castor reports for the tables pinned to it. That is a popularity proxy as fresh as the last sync, and it covers tables only: Castor reports no query count for dashboards.

Nistor matches an asset to a node by its Castor externalId:

Castor asset externalId Nistor node
BigQuery table <project>.<dataset>.<table> BigqueryTable_<project>.<dataset>.<table>
BigQuery column <project>.<dataset>.<table>.<column> a column of that table
MicroStrategy object <project-id>##<object-id> MicrostrategyCube_<object-id>-<project-id>

The tool assumes a BigQuery warehouse. See Google Cloud and MicroStrategy for those nodes.

Permissions#

A Castor API token. Nistor only reads from Castor.

Edit this page on GitLab

↑ ↓ to moveEnter to openEsc to close