Data Sources
A data source is a named connection to an object store or a database. Nodes in a session template point at a data source by name. The Data Sources page is where you register them. Open Platform, then Data Sources, or reach it under Configuration (/admin/configuration/data-sources).
Data sources do not have to live inside the HEAT cluster. If the cluster can reach a remote object store or database over the network, you can register it and direct session data there. HEAT keeps a record of where each piece of data is. The bytes stay in the location you chose.
Built-in data sources
Every HEAT install registers data sources for the common cases, so you can run sessions without an external object store.
| Name | Type | Purpose |
|---|---|---|
HEAT Managed Object Store | S3 | The default target for uploads, node outputs and dashboard data |
HEAT Bulk Analytics DB | PostgreSQL | Registered at startup when the bulk analytics service is deployed |
You can register as many more as your deployment needs.
Supported types
The Type field accepts one of these values.
| Type | Use |
|---|---|
S3 | S3-compatible object stores, including the managed object store, MinIO and Amazon S3 |
AzureBlobStorage | Azure Blob Storage |
PostgreSQL | PostgreSQL databases |
MSSQL | Microsoft SQL Server |
MongoDB | MongoDB |
CosmosDB | Azure Cosmos DB |
Creating a data source
Press New source and fill in the name, a description, the type and the connection string. The name is what nodes and ingests use to refer to the data source, so pick something that says what it holds, for example raw-uploads or dashboard-outputs.
Treat a connection string as a credential. It is accepted once, at creation, and not shown again. On the edit form and to a non-Administrator it reads as hidden. To change the connection details, register a new data source and point the templates that should use it at the new name.
How nodes target a data source
A node names its data source with the dataSourceName key in its configuration. When the key is missing, the node uses the platform default set by the runner_config.default_store setting, which is HEAT Managed Object Store on a fresh install. An ingest also names a data source, and uploads that arrive through it are stored there.
This lets you keep raw input, intermediate results and dashboard data in separate locations by configuring each node in the template.