Node Templates

Node Templates define the type of operation a node can perform within HEAT’s Directed Acyclic Graph (DAG) workflows. They act as reusable blueprints for different kinds of data-processing tasks, such as input, transform, processing, or aggregation. The actual configuration, connections, and scheduling of these operations are determined when you place them in a Session Template, where the Session Template Node references a Node Template.

What Is a Node Template?

A Node Template describes:

Operation Type: For instance, whether it’s an input node that receives data from an Ingest, a transform node that reshapes data, a processing node running algorithms, or an aggregation node combining multiple data streams.
Expected Inputs/Outputs: Defines what kind of data (e.g., JSON, CSV, or custom object) the node expects and what it will produce.
Configuration Schema: Declares any parameters or settings required by the node. This could include a destination Data Source, runtime options, or file format preferences.
Multi-Input Support: Indicates if the node can handle multiple upstream inputs at once (e.g., merging two CSV outputs) or requires a single input only.
Feature/Capability: References the underlying feature or runner (e.g., a custom transformation runner or machine-learning algorithm) that will execute this operation.

You can think of Node Templates as “kinds of operations.” Each node in a Session Template points to a Node Template, along with specifics like how it connects to other nodes and any parameter overrides.

Relationship to Runners

While a Node Template defines what the node is supposed to do conceptually, a runner is what actually executes the logic:

Runners are containerized processes deployed by HEAT to handle one or more Node Template types.
A Node Template typically references a runner name or ID, but it does not itself dictate how the task is performed internally.
This separation of concerns allows you to update or replace the underlying runner without changing every Session Template node.

Defining Node Templates

Like other core HEAT entities (e.g., Projects, Session Templates), Node Templates can be defined or updated:

Via the HEAT API: Programmatically create or modify Node Templates for integration into your CI/CD pipelines.

Using Node Templates in Session Templates

When building a Session Template:

Choose a Node Template: For each step in your data pipeline, select the Node Template that aligns with the operation needed (e.g., an “XML-to-JSON Transform Node”).
Set Configuration: Override or fill in any parameters required by that node (e.g., specifying the Data Source, file formats, or API endpoints).
Link to Dependencies: Indicate which nodes feed data into this node, and which node or nodes it outputs to.

Example Use Cases

Input Node
An input node is typically processed by our Ingests and the payload gets stored in a Data Source for downstream processing.
Transform Node
A Node Template called “transform-csv-biometrics” could read raw CSV input, extract particular columns, and convert to a consistent format for another downstream node.
Processing Node
A Node Template called “process-biometrics” might involve a runner that takes JSON input and applies a machine-learning model to generate predictions or additional metadata.
Aggregation Node
“Combine Metrics” could accept outputs from multiple transform or processing nodes, merging them into a single dataset or summary.

`heat-system-next-dimension` (platform reserved)

The heat-system-next-dimension node template is a platform-reserved aggregation template shipped with Core API presets. It is not meant for general integrator session templates: for normal Next dashboards and custom graphs, use dashboard-v2 on dashboard-utils and the dashboard-v2 upstream contract.

When a platform workflow uses this template, it writes canonical $heat-dataservice JSON to object storage and creates or updates a session dimension with DashboardVersion="Next" for ui/dashboard (layout schema). It does not target ui/legacy. Payload and layout rules match dashboard-v2.

`heat-system-dataservice-envelope` (platform reserved, system-utils only)

The heat-system-dataservice-envelope template is a platform-only transform shipped with Core API presets. It reads one parent node JSON output and writes a new root object whose $heat-dataservice block is assembled entirely from node configuration (groups, channels, optional per-channel data, valueFromBindings with dotted countPath / copyPath into the parent payload, or for shape: "series" a single seriesFromPath dotted path whose value is a JSON array of { "timeMs", "value" } objects). The implementation does not embed upstream-specific field names; templates supply paths and labels.

Use it in a session graph before heat-system-next-dimension when operational or custom JSON must become a Next-ready dataservice. Template names intentionally use the heat-system-* prefix so they are not confused with general-purpose runners such as dashboard-utils / core-utils (json-template, tabular-remap, and similar).

`cluster-usage-history` (platform reserved, system-utils only)

The cluster-usage-history template is a platform-only transform used in the Resource Monitor static pipeline. It requires a cluster-report parent, clones the latest cluster snapshot JSON to its output root, and appends heatClusterUsageHistory.cpuCores and heatClusterUsageHistory.memoryGiB arrays built from up to the last 60 historical cluster-report outputs (oldest to newest), for downstream heat-system-dataservice-envelope channels.

Use it in a session graph after cluster-report and before heat-system-dataservice-envelope when you need bounded usage timelines without changing the KPI snapshot paths on the merged root.

For integrator-owned dashboards and richer shaping, prefer dashboard-v2 on dashboard-utils and the dashboard-v2 upstream contract. The shipped Resource Monitor session template embeds its Next layout; see core/api/Presets/templates/heat-system-resource-monitor-next.json.

`heat-workspace-group-count` (platform reserved, system-utils only)

The heat-workspace-group-count template is a platform-only transform. It reads one parent JSON object (typically the merged workspace root produced upstream, for example after cluster-usage-history), clones that root, and appends a single property heatWorkspaceGroupCountV1 with a versioned categorical summary suitable for Next $heat-dataservice shape: "value" channels and the dashboard BarChart widget.

Node configuration (instance `DefaultConfiguration`)

Property	Required	Description
`itemsPath`	yes	Dotted path from the parent root to a JSON array of objects (e.g. `pods`).
`groupKeyPath`	yes	Dotted path relative to each array element for the bucket key (e.g. `labels.app`). Missing or empty values bucket separately.
`filters`	no	Array of `{ "path", "op", "value" }`. `op` supports `equals` only in v1; `path` is relative to each element (e.g. `namespace`).
`unlabeledBucketKey`	no	Bucket key for items with no group key (default `__unlabeled__`).
`maxGroups`	no	Max distinct groups before remaining items roll into `__other__` (optional).
`sort`	no	`countDesc` (default), `keyAsc`, or `cpuMilliDesc` (order buckets by descending summed CPU millicores; requires `sumContainerCpuRequestsFromPath`).
`barSeriesName`	no	Single-series mode only: Apex `name` for the count series (default `Pods`).
`sumContainerCpuRequestsFromPath`	no	Dotted path relative to each item to a JSON object whose values are Kubernetes CPU request strings (for example `containerCpuRequestsMilli` on `cluster-report` pods). When set in single-series mode, each bar point includes `cpuRequestsMilli` (sum of parsed millicores for that bucket).
`sumContainerMemoryRequestsFromPath`	no	Same pattern for memory request strings (for example `containerMemoryRequests`). Adds `memoryRequestsMi` (MiB) per bar point in single-series mode.
`resourceTotalsAsGroupedBarSeries`	no	When `true` with both sum paths set, `barSeries` is two Apex series (CPU and memory per bucket) instead of one count series. Use with `groupKeyPath` such as `name` for one category per pod (within filters).
`cpuSeriesName`	no	Grouped mode: first series `name` (default `CPU requests (m)`).
`memorySeriesName`	no	Grouped mode: second series `name` (default `Memory requests (MiB)`).

Output: `heatWorkspaceGroupCountV1`

schemaVersion: integer, currently 1.
aggregation: literal countByKey in v1.
itemsPath, groupKeyPath, filters, sort, maxGroups: echo effective settings for auditing.
groups: { "key", "label", "count" }[] sorted per sort.
barSeries: Apex payload. Single-series (default): [{ "name": string, "data": [{ "x", "y", "cpuRequestsMilli"?, "memoryRequestsMi"? }] }] where y is the item count per bucket. Grouped metrics when resourceTotalsAsGroupedBarSeries is true: two entries [{ "name": "<cpu>", "data": [{ "x", "y" }] }, { "name": "<memory>", "data": [...] }] with the same x order; y is millicores or MiB respectively. The Next BarChart uses dual colour and dual y-axes when multiple series are present.

Bind in heat-system-dataservice-envelope with valueFromBindings and copyPath: for example copy heatWorkspaceGroupCountV1.barSeries into a value channel key (e.g. barSeries) so data.value.barSeries is available to BarChart in ui/dashboard.

Best Practices

Keep Node Templates Small & Focused
Each template should handle a single logical step. This approach reduces complexity and makes troubleshooting easier.
Document Expected Inputs/Outputs
Make sure the Node Template’s schema is well-described, so other team members or external integrators know how to pass data between nodes.
Ensure Idempotence
Node Templates that rely on external state or have side effects can lead to unpredictable results. Strive for operations that produce the same output given the same inputs.

Retention (session template nodes)

In a Session Template, each node’s defaultConfiguration may include retention.skipArchive: true so session retention does not trim that node’s output history during ArchiveByAge. Input-type templates are always exempt from archival.

Summary

Node Templates define what an operation is supposed to do (type, expected inputs/outputs, config schema).
Runners handle how that operation is executed at runtime.
By combining Node Templates in a Session Template, you build out an end-to-end data workflow.
They can be created or updated programmatically via the HEAT API and stored for reuse across multiple workflows.

Next Steps

See how Node Templates fit into complete workflows with Session Templates.
Explore how data enters the system through Ingests.
Learn how containers (runners) execute Node Templates in Analytics & Algorithms.