Ingests
An ingest is a managed upload URL. Posting data to it starts a session, or adds to one, without the sender knowing anything about projects or templates.
Each ingest binds a URL path to one project, one session template, one input node in that template and one data source. When a payload arrives, HEAT stores it in the data source, attaches it to the input node of a session and lets the rest of the template run.
An ingest is not the v2 API. The v2 API is the general upload and query surface for integrators who work with sessions directly. An ingest is a fixed door for one kind of payload from one kind of sender.
The current Control Centre at /admin has no Ingests screen. Ingests are created and edited in the legacy console at /main/ingests, and the platform still serves them at runtime. New integrations should prefer the v2 API, which uploads to a session’s input node directly. This page documents the ingest model and the legacy create form.
Where to find it
Open Ingests in the legacy console (/main/ingests). The list shows every ingest and a Create Ingest button.
| Column | Meaning |
|---|---|
| URL | The path the sender posts to, without the /ingest prefix. |
| Project ID | The project new sessions are created in. |
| Template ID | The session template new sessions are created from. |
| Input Node | The node in that template that receives the payload. |
| Behaviour | Create New or Append. |
| Data Type | All, CSV or JSON. |
| Actions | Edit opens the ingest. |
Creating an ingest
Press Create Ingest (/main/ingests/create) and fill in the form.
| Field | Meaning |
|---|---|
| URL | A path that starts with /, for example /my-simulator. Do not include the host name. The sender posts to /ingest followed by this path. |
| Project | The project new sessions are created in. Chosen from a list. |
| Session Template | The template new sessions are created from. Chosen from a list. |
| Input Node | The node that receives the payload. When the chosen template exposes its nodes, this is a list and defaults to the first node. Otherwise type the node name exactly as it appears in the template. |
| Data Source | Where the payload bytes are stored. Chosen from the registered data sources. |
| Creation Behaviour | Create New Session starts a fresh session for every post. Append to Existing Session reuses a session when a lookup matches. |
| Ingest Data Type | All accepts any payload. CSV and JSON accept only that format and enable the lookup used by append. |
| Lookup Key | Shown only for Append to Existing Session. The name of a top-level field in the payload. |
| Allow Anonymous | When ticked, anyone who can reach the URL may post. When clear, the sender must present the ingest’s secret. |
Press Create Ingest. The Control Centre returns to the list when the ingest is saved. If the platform rejects the combination of fields, the reason is shown above the form.
The platform enforces two rules on the combination of fields. Append to Existing Session needs a lookup key. Append to Existing Session is not supported with the All data type, because an untyped payload cannot be searched for the lookup key.
Editing an ingest
Press Edit on a row (/main/ingests/edit/<id>). The edit form has the same fields as the create form, but the project, template and data source are shown as numeric IDs rather than lists. Press Update Ingest to save.
The secret
When Allow Anonymous is clear, the platform generates a long random secret for the ingest at creation. The sender presents it as a bearer token on every request.
Reading and rotating the secret from the Control Centre are not available today. The create and edit forms do not show it. Ask a HEAT engineer to retrieve or rotate the secret for an ingest.
The resulting URL
The ingest is served at https://<your-heat-host>/ingest<URL>. An ingest with URL /my-simulator on heat.example.com listens at https://heat.example.com/ingest/my-simulator.
A GET to that address confirms the ingest exists and reports whether it allows anonymous posts. A POST delivers data.
curl -X POST "https://heat.example.com/ingest/my-simulator" \
-H "Authorization: Bearer <ingest-secret>" \
-H "Content-Type: application/json" \
-d '{"exerciseId": "ex-2041", "playerId": 1234, "actions": []}'Leave out the Authorization header for an ingest that allows anonymous posts. A missing or wrong secret is refused before any data is read.
What happens when data arrives
- The platform checks the secret, unless the ingest allows anonymous posts.
- The payload is parsed as the ingest’s data type. A
CSVingest expects CSV text with a header row, aJSONingest expects a JSON document, and a payload that cannot be parsed is refused. AnAllingest accepts the payload as it is. - A session is chosen. With
Create New Sessionthe platform always creates one from the ingest’s template in the ingest’s project. WithAppend to Existing Sessionthe platform reads the lookup key field from the payload (from the first row of a CSV) and searches the project for a session whose metadata carries the same value. It reuses that session when found. Otherwise it creates a new session and records the lookup value in its metadata so that later posts match. - The payload is written to the ingest’s data source.
- A new node output is recorded on the input node of the session. The rest of the template runs from there.
The response reports the session that received the data. Sessions created by an ingest are named with the time of the first upload.
Append is how several files that belong to one training run land in one session. Every file carries the same value in the lookup key field, for example an exercise identifier, and every post lands in the session created by the first.