Architecture and boundaries
HEAT is a data layer that receives data that a simulator, a sensor or a person has already produced, processes it, and publishes the results as dashboards, reports and files. It never controls the system the data came from.
The HEAT platform is a set of services that run in one Kubernetes cluster, plus a capture application that runs beside the source. Every system outside that cluster reaches HEAT through a versioned, documented external API or via hosted SPAs that use the same external API surface. Everything behind that API is internal to the platform.
Data arrives through the external API, rests in storage you choose, and is processed by short-lived workers that keep nothing when their task ends. People are known to the analysis only by an identifier. Their names live in a separate identity service.
HEAT is not a simulation engine. It does not compute physics, render a scene, author a scenario or send commands to a simulator.
This page describes the boundaries. It does not describe how the services inside the cluster talk to each other, because that is not something an integrator or an operator builds against.
The shape of the system
The external surface
An external system is anything that runs outside HEAT’s Kubernetes cluster. That includes HEAT Capture, a simulator integration, a training management system, a script and a browser. All of them use the same small surface.
| Surface | Used for | Documented in |
|---|---|---|
| HEAT v2 API | Creating sessions, uploading data, checking progress, reading results, exporting reports | The API reference and the integration walk-through |
| HEAT Auth sign-in and token routes | Signing in, and issuing long-lived tokens for unattended devices | Offline tokens |
| The HEAT dashboard and the Control Centre | People reading results and operating the environment | The Users and Administrators sections |
The external surface is a contract.
- It is versioned. The version is part of every route, as in
/api/v2/. A published request or response shape is not changed. New capability arrives as new routes or fields, or as a new version beside the old one. - It is documented twice. This site is the living reference and changes with each release. Versioned interface control documents (ICDs) for the external interfaces are available on request, for programmes that need a fixed document to accredit or integrate against.
- Every call is authenticated. A call carries a token from HEAT Auth, and each route states the roles it accepts. A capture device uses an offline token, which can upload and read but cannot administer users or issue tokens.
- It is the only supported way in. HEAT Capture has no private channel. It uses the same v2 routes as your own software.
Getting data in
Data enters through the upload route of the v2 API. Software creates a Session from a session template and sends bytes to an input node.
Switch the ingestion queue on for any Project that receives live or high-volume data. The API then accepts each upload at once, and the queue writes many small uploads to storage as fewer, larger objects. This is the highest-throughput path, and it is the one HEAT Capture is designed for. See Projects for the control.
Internal interfaces
The services inside the cluster have their own interfaces for coordinating sessions, handing tasks to runners and moving uploads through the queue.
These interfaces are not a contract. They change from release to release as the platform evolves, they are not versioned for outside callers, and they are not intended to be reachable from outside the cluster. This documentation does not describe them.
Some environments make an internal interface reachable for operator tooling or diagnosis. That does not make it an integration surface. Software built against an internal interface can stop working on any upgrade. Build against the v2 API.
The source boundary
Data flows one way, from the source into HEAT. HEAT reads what a source writes, sends or broadcasts. No external or internal interface sends a command back to a simulator.
HEAT sits outside the simulation loop. Its processing time never affects a simulator’s frame rate, and a fault in the HEAT platform does not stop a training run. A new source needs a connector on the outside, not a change to the platform. See Connecting a simulator or sensor.
Data at rest and data in processing
HEAT separates where data rests from where it is worked on.
| Place | What it holds | For how long |
|---|---|---|
| Your data source | Every upload and every node output. This is the data. | Until your retention policy or your own storage rules remove it |
| Ingestion queue | Uploads that have been accepted and not yet written to the data source | Moments, in normal running |
| Runners | The inputs of one Task, in the memory and scratch space of a short-lived worker | For the life of that task |
| Platform database | The records that describe projects, templates, sessions and outputs, including storage paths and session metadata. No payloads. | For the life of the session |
| Identity store | Accounts, roles and personal details | Until the account is removed |
| Bulk analytics database (optional) | Rows copied from tabular outputs, only when a template uses Bulk analytics | Until the session is deleted |
Your own storage
A data source can be an S3-compatible object store that you host and control. HEAT then holds no copy of the data at rest. It reads an object when a task or a dashboard needs it and writes each result back to the same store.
Two conditions apply. The store must be reachable from the cluster with low latency, because every task reads and writes through it. It must also sustain the write rate of your captures. A slow store slows every session that uses it.
Your store stays inside your own accreditation boundary. Encryption at rest, backup, access logging and lifecycle rules are yours to set. Removing HEAT’s access to the store removes HEAT’s access to the data. See Where your data lives.
Short-lived workers
All processing happens in runners inside the cluster. A runner claims one task, reads that task’s inputs from the data source, stores one output and moves on. It holds no state between tasks, and the platform starts and stops runner pods as the workload changes.
A runner sees the inputs of the task it claimed and nothing else. It does not call the identity service, so a processing step cannot learn who a person is.
Every node output is immutable. Reprocessing adds a new output and leaves the old one in place, so a result can always be traced to the data and the template version that produced it.
Pseudonymisation
HEAT separates who a person is from how the analysis refers to them. This is a design rule of the platform and not a setting.
| Layer | Knows a person as | Holds personal details |
|---|---|---|
| Identity service (HEAT Auth) | An account with an opaque identifier | Yes. Email address, name, title and company. Nowhere else. |
| Platform, runners and stored data | A number inside the environment, linked to the opaque identifier | No |
| External API and the dashboards | The opaque identifier, with a name added at the moment of display | Only in the response being sent |
The platform follows these rules.
- Personal details are stored once. HEAT Auth is the only store of names and email addresses. Sessions, dimensions, node outputs and reports carry identifiers.
- Names are joined at display time. When someone opens a result, the API asks HEAT Auth for the names behind the identifiers, using the viewer’s own token. HEAT Auth decides what that viewer may see. The answer is returned and not kept.
- No copy is made on the way. The platform never writes a name into session metadata, node configuration, node outputs or its database, and it does not cache names between requests or between viewers.
- Processing never sees a name. Runners work with identifiers only.
Four properties follow, and each can be checked against a running environment.
- Stored and exported analysis data carries no personal details that HEAT added, so it can be copied, backed up or shared without HEAT having placed a name in it.
- A person is forgotten by removing one account. There are no names to find and erase in the analysis data.
- A leak of the data source exposes performance data against meaningless identifiers. Linking it to people needs the identity service as well.
- With the identity service switched off or out of reach, HEAT cannot connect any result to a person.
Controlled information
Pseudonymisation protects identity. Controlled Unclassified Information (CUI) and other sensitive content, such as the performance parameters of a platform, need a second set of controls.
HEAT does not inspect the content of your data, classify it or mask fields on its own. It gives the template author and the operator a framework in which the secure choice is the ordinary one. The platform supplies the boundaries above. The design of the session template and the way the API is used complete the picture.
| Decision | Made by | Effect |
|---|---|---|
| Identify participants by their HEAT Auth identifier. Keep names out of session names, session metadata and uploaded files. | Integrator | The pseudonymisation model covers everything HEAT stores |
| Give sensitive inputs their own data source, named on the input node | Template author | Raw controlled data rests in a store with its own access rules, apart from results |
| Publish derived measures, not raw parameters | Template author | A dashboard shows only the channels the template publishes. A report holds only the columns its definition names. A score can be released where the parameter behind it cannot. |
| Set a retention policy, with an inputs stage | Template author | Raw inputs are removed after an idle period while dashboards and results remain |
| Use offline tokens on capture devices, and roles for people | Operator | A device can upload and nothing more. A Trainee opens only their own sessions. |
| Run a separate environment for each audience that must be kept apart | Operator | Environments share nothing. Each can run fully air-gapped. |
| Delete a session when its data must go | Operator | The session record, its outputs and its stored objects are removed together |
See Session retention for the stages, and Where your data lives for export and removal.
Releasing a subset
What a viewer can see is decided by the session template, so a release rule is a matter of template implementation. One session can publish several dimensions, each with its own layout, its own data and its own list of users. A template can publish a dashboard for each participant that carries only that participant’s data points, beside a fuller dashboard for instructors, and it can leave any data point out of a dashboard altogether.
Identity is collated only at the point of request, for an authenticated user, and only for the people that user is allowed to see. The same stored result can therefore be shown with names to one audience and with identifiers alone to another, without a second copy of the data.
Access inside an environment
Roles apply to the whole environment. An Instructor, an Administrator or a Developer can open every session, and a Trainee can open the sessions they took part in. See Signing in and roles.
A project organises sessions and templates. It does not restrict who can see them. The unit of isolation is the environment.
Timing and scaling
HEAT works in near real time. HEAT Capture streams while a training run is in progress, nodes that support incremental processing handle new data as it arrives, and a dashboard updates as outputs land. HEAT states no latency figure, because the time from an event to a chart depends on the session template and on the runner capacity of the environment.
Processing capacity grows by adding runner pods. The platform scales each runner type within a range that the runner declares and an operator can override. The default range is two to four pods. A project with the ingestion queue switched on buffers its uploads independently of other projects. Storage grows with the object store you chose.