Build your first data flow

A data flow in HEAT is a session template plus a way to get data into it. This page builds one twice on the same ten-row CSV. Track A uses a template the platform ships, so you author nothing. Track B authors the template by hand, so you see every part you will later change.

Both tracks end in the same place: a Session whose nodes read Success in the Control Centre and a dashboard you can open. Do Track A first. It confirms the environment works before you author anything.

Before you start

You need a HEAT environment you can sign in to with an Administrator account, and the HEAT Control Centre (formerly Cluster Manager, still served at /main and /admin). You also need a Project to create sessions in. Any existing project will do. Otherwise create one under Projects.

Save this file as sample.csv. It has a millisecond time column and two numeric columns.


time_ms,speed_kph,altitude_m
0,0,120
1000,12,121
2000,25,124
3000,38,130
4000,47,139
5000,52,151
6000,55,164
7000,54,176
8000,49,185
9000,41,190

Track A: the shipped preset

The platform ships a session template named system-tabular-dashboard-autodetect-next. It takes any CSV, works out which column is time and which columns are numeric series, and opens a dashboard with a suggested layout. Every node in it runs on system-utils, the platform’s built-in runner hosted inside the Core API, so it works on a minimal install with no extra features enabled.

What the preset contains

The template has three nodes in a straight line.

Node name	Node template	What it does
`tabular-upload`	`input-node`	Receives the upload. Every upload becomes one node output on this node.
`tabular-autodetect`	`system-tabular-to-dataservice`	Reads the CSV outputs, builds a data service payload and emits a suggested layout.
`tabular-dashboard`	`heat-system-next-dimension`	Publishes a Dimension for the Next dashboard using that payload and layout.

The configuration on tabular-autodetect is what makes the template work on unknown files.

Key	Value	Effect
`autoDetectTimeColumn`	`true`	Picks the time column by sampling rows instead of requiring `timeColumn`.
`emitSuggestedLayout`	`true`	Writes a layout with one chart per numeric column into the output for the dashboard node to use.
`whenNoOutputs`	`"complete"`	Succeeds quietly when the input node has no uploads yet.
`realmPerFile`	`false`	All uploaded files land in one Realm instead of one realm per file.
`defaultRealm`	`"default"`	The realm name used when no realm column exists.
`maxAutodetectSampleRows`	`200`	How many rows the detector reads before deciding.

The configuration on tabular-dashboard names the storage and the dashboard.

Key	Value	Effect
`dataSourceName`	`"HEAT Managed Object Store"`	Where the published payload is written.
`dashboardName`	`"CSV autodetect dashboard"`	The title the dashboard shows.
`autoCreateScenarioInstance`	`true`	Creates the scenario record a dashboard needs when the upload did not carry one.
`scenarioName`	`"csv-autodetect"`	The name of that scenario record.
`layoutFromParentOutput`	`true`	Uses the suggested layout from the parent node instead of the `layoutConfiguration` in the template.
`layoutConfiguration`	An empty layout	The fallback when the parent emits no layout. It declares the `default` realm and global playback controls with no rows.

Four steps take the preset from a file on your disk to a dashboard.

Create an ingest

An Ingest is a managed upload URL bound to a project, a session template and an input node. Open Ingests in the Control Centre and press Create Ingest.

Field	Value
URL	`/first-flow`
Project	Your project
Session Template	`system-tabular-dashboard-autodetect-next`
Input Node	`tabular-upload`
Data Source	`HEAT Managed Object Store`
Creation Behaviour	`Create New Session`
Ingest Data Type	`CSV`
Allow Anonymous	Ticked, for this exercise only

Leave Allow Anonymous clear on any ingest that outlives this exercise and use the ingest secret instead.

Post the CSV


curl -X POST "https://heat.example.com/ingest/first-flow" \
  -H "Content-Type: text/csv" \
  --data-binary @sample.csv

The response names the session that received the file. Copy its id.

Watch the nodes

Open Sessions in the Control Centre, find the session and open the Nodes tab. Within a minute tabular-upload, tabular-autodetect and tabular-dashboard each show Success. If a node shows Failed, its status details say why. The usual cause on a fresh install is a data source name that does not match.

Open the dashboard

Sign in to the HEAT dashboard and open Sessions. The new row is named after the time of the upload. Open it. You see one chart for speed_kph and one for altitude_m on a shared time axis, with playback controls at the top.

That is a complete HEAT data flow with nothing authored. Everything in Track B replaces one of these parts with something you control.

Track B: your own template

Track B builds the same result from three nodes you choose, with a layout you write. It uses two platform runners that are not built into the Core API, so the first step enables them.

Enable the runners

Open Features in the Control Centre. Confirm that Core Utils and Dashboard Tools are registered and switched on. Core Utils provides the tabular-to-dataservice node template. Dashboard Tools provides dashboard-v2, the runner-hosted dashboard publisher for the Next dashboard (its container image is dashboard-utils).

If either feature is missing, the environment has not had its manifest uploaded. Ask the operator.

Author the template

Save this as my-first-flow.json. The shape is the same as the preset. The template has a Name, a Description, a template-level Configuration and a Nodes array. Each node has a Name, a NodeTemplateName, a DefaultConfiguration and a list of Parents by name.


{
  "$schema": "https://heatcommon.blob.core.windows.net/schema/session-template/v1.json",
  "TemplateVersion": "1.0.0",
  "Name": "my-first-flow",
  "Description": "CSV upload on csv-input with time_ms, speed_kph and altitude_m columns. tabular-to-dataservice builds channels, dashboard-v2 publishes a Next dimension with a hand-written layout.",
  "Configuration": {},
  "Nodes": [
    {
      "Name": "csv-input",
      "NodeTemplateName": "input-node",
      "DefaultConfiguration": {
        "dataSourceName": "HEAT Managed Object Store"
      },
      "Parents": []
    },
    {
      "Name": "to-dataservice",
      "NodeTemplateName": "tabular-to-dataservice",
      "DefaultConfiguration": {
        "inputFormat": "csv",
        "timeColumn": "time_ms",
        "timeColumnIsMs": true,
        "baseCapturedAt": "2026-01-01T00:00:00Z",
        "defaultRealm": "default",
        "groups": [
          { "id": "metrics", "name": "Metrics" }
        ],
        "defaultGroupId": "metrics"
      },
      "Parents": ["csv-input"]
    },
    {
      "Name": "publish",
      "NodeTemplateName": "dashboard-v2",
      "DefaultConfiguration": {
        "dataSourceName": "HEAT Managed Object Store",
        "dashboardName": "My first flow",
        "dashboardVersion": "Next",
        "autoCreateScenarioInstance": true,
        "scenarioName": "my-first-flow",
        "layoutConfiguration": {
          "$schema": "https://heatvr.io/layout-schema.json",
          "version": "1.0.0",
          "configuration": {
            "enableGlobalPlaybackControls": true,
            "defaultRealm": "default",
            "showRealmFilterDropdown": false
          },
          "realms": ["default"],
          "components": {
            "rows": [
              {
                "cols": [
                  {
                    "component": "ComposableChart",
                    "colspan": 12,
                    "configuration": {
                      "name": "speed-and-altitude",
                      "titleContent": "Speed and altitude",
                      "shouldRenderTitle": true,
                      "enablePlaybackControls": true,
                      "channels": ["speed_kph", "altitude_m"],
                      "composableChartItem": {
                        "anchor": "time",
                        "elements": [
                          {
                            "type": "legend",
                            "order": 0,
                            "height": 40,
                            "legendItems": [
                              { "id": "speed", "label": "Speed (km/h)", "color": "#81842C", "swatch": "line" },
                              { "id": "altitude", "label": "Altitude (m)", "color": "#0B5370", "swatch": "line" }
                            ]
                          },
                          {
                            "type": "area",
                            "order": 1,
                            "label": "",
                            "height": 220,
                            "series": [
                              {
                                "id": "speed",
                                "label": "Speed",
                                "channel": "default:speed_kph:speed_kph",
                                "color": "#81842C",
                                "render": "line",
                                "unit": "km/h"
                              },
                              {
                                "id": "altitude",
                                "label": "Altitude",
                                "channel": "default:altitude_m:altitude_m",
                                "color": "#0B5370",
                                "render": "line",
                                "unit": "m"
                              }
                            ]
                          },
                          {
                            "type": "playback",
                            "order": 2,
                            "label": "Seek",
                            "height": 56
                          }
                        ]
                      }
                    }
                  }
                ]
              },
              {
                "cols": [
                  {
                    "component": "StatsSummary",
                    "colspan": 12,
                    "configuration": {
                      "name": "run-summary",
                      "titleContent": "Run summary",
                      "shouldRenderTitle": true,
                      "channels": ["run_summary"],
                      "statsItems": {
                        "position": 1,
                        "columnCount": 2,
                        "columns": [
                          { "label": "Top speed", "name": "max_speed_kph", "unit": "km/h" },
                          { "label": "Peak altitude", "name": "max_altitude_m", "unit": "m" }
                        ]
                      }
                    }
                  }
                ]
              }
            ]
          }
        }
      },
      "Parents": ["to-dataservice"]
    }
  ]
}

Four details in this file matter more than the rest.

The input node is named csv-input. That name is the convention for CSV uploads and tells readers what the template expects. It also carries dataSourceName so that uploads through the v2 API know where to write.
tabular-to-dataservice turns every numeric column into a series Channel. The channel id is the lowercased header, so the columns become speed_kph and altitude_m. Reserved columns such as time_ms are skipped.
ComposableChart slices bind to channels with a URI of the form realm:channelId:name. The name of an auto-discovered channel equals its id, which is why the URIs repeat the header.
The StatsSummary row is bound to a channel named run_summary that nothing produces yet. It renders the empty state until the iterate step below adds a node that emits it. Leaving it in now shows how a layout evolves ahead of the data.

Import the template

Templates are imported as JSON files. On import, HEAT reads the Name, checks that every NodeTemplateName is registered on the environment, checks that Parents form a graph with no cycles and stores the template environment-wide. The rules, including how a same-name update is treated, are on the Control Centre Templates page.

Uploading a template file from the Control Centre is not available today. The Templates page lists templates and its create form is disabled. Hand the file to a HEAT engineer or an integrator with import access, then confirm that my-first-flow appears in the list at /main/templates/manage.

An import fails with “unknown node templates” when Core Utils or Dashboard Tools is not enabled. Go back to the first step.

Get data in

Choose one of two doors. The ingest is simpler. The v2 API is what your own software will use.

Ingest

Create a second ingest exactly as in Track A, with URL /my-first-flow, Session Template my-first-flow and Input Node csv-input. Then post the file.


curl -X POST "https://heat.example.com/ingest/my-first-flow" \
  -H "Content-Type: text/csv" \
  --data-binary @sample.csv

Verify

Open the session in the Control Centre Sessions list. On the Nodes tab, csv-input, to-dataservice and publish show Success. The output of to-dataservice is a JSON document with a $heat-dataservice key, one realm named default and two channels.

Then open the session in the HEAT dashboard. The chart shows both series with a legend and a seek bar. The summary row shows “No data”. That is expected at this stage.

If to-dataservice fails with a message about the time column, check that the header reads exactly time_ms. If publish fails, the usual causes are a layoutConfiguration without components.rows or a dataSourceName that does not match a registered data source.

Iterate

The template is now yours to change. Three changes show the shape of everyday work.

Change the layout and reprocess one node. Edit titleContent, change a colour, or add a second area slice. Because the graph has not changed, the file can be imported again as an update with "updateIfExists": true at the top level. Then ask a HEAT engineer to reprocess the publish node only. Triggering a reprocess from the Control Centre is not available today. Upstream outputs are immutable, so the reprocess re-reads the same to-dataservice output and publishes a fresh dimension, which appears as a new row on the node’s outputs list.

Add a transform. Put tabular-remap between csv-input and to-dataservice to rename or derive columns, or tabular-query to filter and aggregate with SQL. Adding a node changes the graph, so save the file under a new name such as my-first-flow-r2 and point the ingest at it. This is the rule for every graph change and it is explained under Authoring session templates.

Fill the summary with Arbex. tabular-to-dataservice emits only series channels, and StatsSummary needs a value channel. Add a system-arbex-js node that reads the CSV, computes max_speed_kph and max_altitude_m and returns a $heat-dataservice payload with a run_summary channel of shape value. Merge it with the series payload through heat-dataservice-merge before publish. Once you are comfortable, the same Arbex node can build the whole layout with heat.layout and the dimension node can take it through layoutFromParentOutput. See Arbex.

Where to go next

You have seen the whole loop: template, import, data in, nodes to Success, dashboard. The rest of the Developers section goes deeper on each part.

HEAT documentation team. Last updated 2 October 2026.

Opens your browser's print dialog. Choose "Save as PDF" as the destination.