Interface control document

1. Identification

2. Scope

This document defines the Tobii Stream Engine surface the module actually calls, the callback data it consumes, the two frame shapes it publishes to HEAT Capture, and the H.264 camera stream contract.

Audit finding TOBII-02 recorded that the module’s data page described a video and stream contract without specifying it as an interface. Section 7 is that specification. It is constrained to what source in this repository shows, and section 7.6 states explicitly which stream parameters this repository does not define. Where a parameter is decided by the vendor runtime or by the Media Foundation encoder selected at run time, that is said rather than filled in with a plausible value.

The module reads measurement data and writes device configuration. It does not expose any remote or network interface of its own; every peer call is an in-process P/Invoke.

3. Interface inventory

3.1 Native entry points used

Every entry point below is declared with CallingConvention.Cdecl against library name tobii_stream_engine. The DLL is vendored in this repository and copied to the package output root, from where the module loader resolves it.

The bindings in this repository also declare presence, user presence, user position guide, gaze origin, normalised eye position, track box, digital syncport, illumination mode, face type, diagnostics image, power save and remote wake entry points. None of them is called by the module in this revision. They are dormant binding surface, not part of this interface.

The module never calls tobii_get_api_version, so it neither reads nor asserts the version of the Stream Engine library it loads.

4. IF-CMTOBII-01 Discovery, licensing and connection

4.1 Addressing and device selection

The connected URL is logged twice during startup, as Connecting to eye tracker: {Url} and then Device context created for {Url}.

4.2 Authentication and authorisation

There is no authentication of HEAT Capture to the tracker and none of the tracker to HEAT Capture. Any local Tobii device that enumeration returns is accepted.

What the LicenceFile setting provides is authorisation for privileged Stream Engine feature groups, not identity. The file is read from the module assembly’s own directory joined with the setting value, decoded as UTF-16, and passed as a tobii_license_key_t whose size_in_bytes is the character count times two.

The licence levels the vendor headers require for the calls this module makes:

LicenceFile defaults to an empty string. With the default, the joined path resolves to the module assembly’s own directory, and the file read is attempted against a directory. The module performs no existence check before that read, unlike the calibration path which does check. The resulting I/O exception propagates out of InitialiseModule, where the SDK catches it and logs {Module} InitialiseModule failed., and the module does not start. A missing or unset licence file is therefore a hard startup failure reported as an exception, not a diagnosable configuration error.

4.3 Connection lifecycle

InitialiseModule runs, in order: create the API context, enumerate trackers, read the licence file and connect to the first tracker, configure the display area (IF-CMTOBII-04), then apply the calibration blob. All of it is bounded by the SDK StartupTimeout. The module does not override that property, so the bound is the SDK default of 5 seconds, covering enumeration, connection, licence file I/O, three device configuration round trips and a calibration apply.

StartCapture assigns the five callback delegates to fields so they are not collected while native code holds them, issues the four data subscriptions, optionally starts the video path (IF-CMTOBII-03), and then starts the pump thread TobiiCallbackThread.

Subscription failures do not fail the start. Each subscribe is wrapped in a helper that logs at critical level and returns, so StartCapture returns true even when every subscription was rejected. The observable result is a module that runs and publishes frames containing only default values. Distinguish this case by the presence of Failed subscribing to ... critical log lines.

EndCapture cancels the pump token and joins the pump thread for up to 2 seconds, cancels the encode token, completes the camera queue and joins the encode thread for up to 2 seconds, logging Encode thread did not stop in time if that join fails. It then unsubscribes the four data streams, unsubscribes the camera, disconnects, destroys the API context, and zeroes both handles. Failures at this stage are logged as errors and do not stop teardown.

The SDK clears its initialised flag on stop, so a subsequent start re-runs InitialiseModule and rebuilds both contexts. Enumeration, connection, display-area configuration and calibration are therefore repeated per capture session, not once per process.

The camera unsubscribe is guarded by the current value of CaptureVideoFeed rather than by the value that was in effect when the subscription was made. Toggling the setting during a session makes teardown either skip a live camera subscription or attempt to unsubscribe one that was never made.

4.4 Reconnect

Reconnect is handled inside the pump loop, not by a connection supervisor.

The loop also exits if the device context has been zeroed, which happens only during teardown.

5. IF-CMTOBII-02 Gaze, eye-openness and head-pose subscriptions

5.1 Delivery model

Four callbacks are registered and all are invoked from the module’s own pump thread during tobii_device_process_callbacks. Each callback receives its vendor struct by reference, validates per-field validity flags, and writes into three long-lived component objects held by the module. No queue, ring buffer or timestamp history is kept: each callback overwrites the retained value.

The callback bodies are individually wrapped in try/catch and log Exception in OnGazeDataCallback, Exception in OnGazePointCallback, Exception in OnHeadPoseCallback or Exception in OnEyePpennessCallback. The last of those log strings contains a typo in this revision and is reproduced here as it appears, because log-based diagnostics have to match the emitted text.

Each callback returns immediately without writing anything if the device context has been zeroed.

5.2 Wire format consumed

The module reads no timestamp from any of these structs. timestamp_tracker_us and timestamp_system_us on gaze data and eye openness, and timestamp_us on gaze point and head pose, are all ignored. It also declares but never reads gaze_point_validity, gaze_point_from_eye_tracker_mm_xyz, gaze_point_on_display_normalized_xy, eyeball_center_validity and eyeball_center_from_eye_tracker_mm_xyz: two local validity variables are computed from the first and fourth of those and then never used.

6. IF-CMTOBII-05 Gaze frame publication and data model reference

6.1 Frame shape

Every capture cycle builds and returns one SimulationFrame with exactly one entity.

The entity id is derived from a fresh GUID, so it is a fresh unpredictable value on every host process start. It is stable for the lifetime of a module instance, including across stop and restart of capture within one process, but it is not reproducible and carries no relationship to the device or to the entity name. Correlating entities across sessions requires the entity name, not the id.

HandleCapture never returns null, so a frame is published on every cycle whether or not any callback has fired. Before the first callback the frame carries default component values, and the components are never reset at start or end of capture, so the first frames of a second session in the same process carry the previous session’s last values. There is no timestamp, sequence number or validity flag on the published data to distinguish a fresh reading from a retained one; only left_eye_occluded and right_eye_occluded carry any liveness signal, and those default to false.

HandleCapture returns the retained component objects themselves rather than copies, so serialisation by the SDK reads objects that the pump thread is concurrently mutating, with no lock or exchange between the two threads. A frame can therefore mix fields from two different device samples.

6.2 Published fields

left_eyelid_openness and right_eyelid_openness are the only readings written without checking their validity flag. A consumer must gate them on the paired quality field, which is the binary 1.0 or 0.0 carrying that flag, and not treat a 0.0 quality reading as a measured value of an eyelid.

The pose is rebuilt from an empty builder on each head-pose callback and assigned only if position or rotation was valid that callback. When only one of the two is valid, the other half of the pose is the builder default rather than its previous value, so a position update during a rotation dropout publishes a default orientation rather than the last known one.

Not populated by this module: gaze_direction, left_gaze_direction, right_gaze_direction, combined_eye_position, gaze_point, blink, has_glasses, and every gaze quality field. SpatialComponent carries only pose; the module sets no image, velocity or other spatial field. PupillometryDataComponent combined fields and other_pupil_data are not populated.

6.3 Cadence

Gaze publication is driven entirely by the SDK capture loop, so it runs at Interval, default 1000 ms, re-read every cycle. The device is asked to run at 33 Hz (IF-CMTOBII-04), so at default settings the callbacks update the retained values roughly 33 times for every frame published, and 32 of those 33 samples are discarded. The module offers no native-rate publication path and no per-sample buffering. To approach the device rate, Interval must be lowered.

7. IF-CMTOBII-03 and IF-CMTOBII-06 Camera stream

Active only when CaptureVideoFeed is true, which is the default.

7.1 Ingest

tobii_primary_camera_image_subscribe is called with a null user-data pointer. The callback receives tobii_primary_camera_image_t with timestamp_us, bits_per_pixel, width, height, stride, data_size and data.

The vendor header documents timestamp_us as having an undefined epoch, useful only for measuring elapsed time between a pair of values. The module does not use it. It stamps each frame with DateTimeOffset.UtcNow at callback receipt instead, and that host-clock value is what reaches the consumer as capture_time. Video timestamps are therefore not comparable with the tracker clock, and they include the callback delivery latency of the pump thread.

Accepted frames must have non-zero data_size, width and height, bits_per_pixel equal to 8, and a stride that is not smaller than width; a stride of zero or less is treated as equal to width. Any other pixel format logs Unsupported camera pixel format: {Bpp}bpp, video disabled once and disables video for the session. The pixel data is copied out of the native buffer before the callback returns.

7.2 Queue and back pressure

7.3 Encode

A dedicated background thread named TobiiEncodeThread drains the queue. The encoder is created lazily on the first frame, because the frame dimensions are not known until then, and it is created, used and disposed only on that thread.

The initialisation log line is H264 encoder: backend={Backend} {W}x{H}@{Fps} and reports the 16-aligned encode size, not the camera’s native size.

7.4 Published frame shape

One SimulationFrame per completed access unit, published immediately through the SDK Publish path from the encode thread. It is not produced by HandleCapture and is not gated by Interval.

There is no keyframe flag on the message. Identifying an IDR requires inspecting NAL unit types in video_data. A consumer must decode in arrival order per stream name and cannot begin decoding before an IDR arrives.

7.5 Cadence

Video cadence is the rate at which the device delivers camera frames, reduced by any frames dropped at ingest, at the queue or by the encoder. The module never resamples or pads, so the emitted rate is not held to the declared frame_rate of 33. The declared value is a constant in source and is not measured, so it should be treated as a nominal hint rather than as the stream’s true rate.

7.6 Parameters this repository does not define

The following are outside what source in this repository fixes, and are recorded here as undetermined rather than assumed:

• Camera native resolution and frame rate. The module accepts whatever the first callback reports and derives its encode size from that. No resolution or rate is asserted anywhere in the module. The authority is the connected Tobii device and its runtime.
• Bitstream framing and parameter-set placement. The module sets no bitstream-format attribute on either media type. Annex-B framing with SPS and PPS present on IDR frames is asserted by the encoder’s own documentation comment and by the module’s design specification as the behaviour of the Media Foundation H.264 transform. It is neither configured nor verified by code in this repository, and the transform chosen varies by machine because hardware encoders are preferred over the Microsoft software fallback. Verify framing against the actual capture before relying on it.
• Profile and level. Not set. Whatever the selected transform defaults to.
• Achieved bitrate. 2 000 000 is a requested average and a best-effort constant-bitrate request. A hardware transform may reject the rate-control mode, in which case its own default applies.
• Keyframe interval. GOP size 33 is requested best-effort, and forced keyframes are additionally requested on queue overflow, which itself may be rejected and is then logged as ForceKeyframe request rejected.
• Colour metadata. None is set on either media type. The source is 8-bit greyscale and the chroma plane is synthetic neutral, so colour primaries, transfer characteristics and matrix coefficients are whatever the transform writes by default.
• Access-unit-to-frame ratio. Timestamp carry assumes one output access unit per submitted frame. If a transform emits an additional standalone output, the FIFO shifts and later access units carry the timestamp of an earlier frame. When the timestamp queue is empty, the access unit is published with a capture_time of 0.

8. IF-CMTOBII-04 Device configuration

Applied once per capture session during InitialiseModule, before any subscription.

1. tobii_get_geometry_mounting reads the tracker’s own mounting geometry. Its log contains Tracker geometry, {Guides}, {Width}mm and {Angle}deg. Failure logs Failed retrieving geometry mounting and aborts initialisation.
2. tobii_calculate_display_area_basic combines ScreenWidth, ScreenHeight and TrackerOffsetX with that geometry. Failure logs Failed calculating display area and aborts initialisation.
3. tobii_set_display_area applies the result, and the three computed corners are logged. Failure logs Failed setting display area and aborts initialisation.
4. tobii_set_output_frequency requests 33 Hz. TOBII_ERROR_INSUFFICIENT_LICENSE logs a warning stating the device will run at its default frequency; any other error logs Failed setting output frequency. Neither aborts initialisation, so a session can run at a device frequency other than 33 Hz while the module still declares frame_rate 33 on video and still assumes 33 fps for encoder timing.
5. tobii_calibration_apply applies the blob at CalibrationBlobPath, resolved relative to the module assembly directory and defaulting to data/calib_default.blob, a 345-byte file bundled in the package. A missing file logs a message containing No calibration blob found at '{Path}' and proceeding without applying calibration, then continues uncalibrated. An apply failure logs Failed applying calibration blob and also continues.

The same bundled default blob is applied on every session. The module provides no per-user or per-session calibration step, so gaze accuracy reflects that default calibration and not the current operator.

9. Units and conventions

10. Configuration affecting the interfaces

The output frequency request, the video frame rate, the encoder bitrate, the GOP size, the queue capacity, the thread join timeouts and the reconnect behaviour are all compile-time constants with no setting. Settings changed mid-session have no effect on any subscription, because subscriptions are made only at StartCapture.

See Manage capture modules for the generic settings and the SDK ICD for the lifecycle and publication contract this module implements.

11. Error behaviour

No error condition in this module raises a HEAT-visible fault, and none stops the capture session. Every failure short of a startup failure degrades silently into missing or frozen data, so log inspection is the only reliable way to establish that a session’s data is trustworthy.

12. Compatibility and versioning

There is no version negotiation on either side. The module does not query the Stream Engine version and does not read the device model or firmware. Compatibility rests on the vendored tobii_stream_engine.dll in this repository matching the layout of the binding structs alongside it; a vendor DLL swap that changes any consumed struct layout produces misread fields rather than an error, because the bindings are sequential-layout structs marshalled directly.

Device requirements that follow from source rather than from documentation: the tracker must be screen-based, since the module uses the screen-based API and not the wearable API whose bindings are also present but unused; it must supply an 8-bit greyscale primary camera image for video to work; and the licence must reach at least Professional for gaze data and Internal for the camera image.

The host contract is HEAT.Capture.ModuleSDK 2.0.3, targeting net10.0. The module is Windows-only in practice: Media Foundation for encoding and the vendored Windows Stream Engine DLL.

Audit finding TOBII-01 applies to pinning an artifact to this revision. In this revision, non-main builds select a full SemVer package version but the generated release location and tag use only majorMinorPatch, so a prerelease package is published under a tag that looks like a stable release and collides with the eventual stable release of the same version. Pin by commit, and verify the package version recorded inside the artifact rather than trusting the release tag, when confirming that a deployed module is this revision.

13. Security considerations

Gaze, pupillometry, head pose and the infrared camera image are all personal data. The camera feed in particular is imagery of a person’s eyes and face region, and gaze data is behavioural biometric data from which attention and identity inferences can be drawn. Both are published into HEAT Capture and persisted with the session.

Restrict access to the workstation, the module directory and stored captures according to the deployment’s own data-handling policy, and treat any exported session containing camera.ir as containing identifiable imagery.

14. Conformance

1. Place a valid Tobii licence file in the module directory and set LicenceFile to it. Confirm the log records License file loaded from '{Path}' and Device context created for {Url}. Confirm no Failed subscribing to ... critical line appears for any of gaze data, eye openness, head pose or gaze point.
2. Confirm the startup log records the tracker geometry line, the three display-area corners, and either Output frequency set to 33Hz or the insufficient-licence warning. If the warning appears, record the device’s actual frequency, because the declared video frame_rate of 33 will be wrong for that session.
3. Confirm one entity named Tobii_Data appears in the captured data, and that both frame shapes arrive on the same entity id: gaze frames carrying gaze, pupillometry and spatial components, and video frames carrying only a stream component.
4. With the operator looking at known screen corners, confirm the sign convention of gaze_screen_point. Looking at the top of the screen should be checked against the y value: a value near 0 confirms the top-down Tobii convention described in section 9, and near 1 would contradict it. Fix downstream interpretation to match what you observe, not to what the proto field documents.
5. Confirm eyelid openness is only trusted when its quality field is 1.0. Cover one eye and check that the corresponding quality drops to 0.0 while the openness field still carries a number.
6. With Interval at the default, confirm gaze frames arrive at about 1 Hz and not at the device rate. Lower Interval and confirm the rate follows it.
7. With CaptureVideoFeed on, dump the video_data of consecutive camera.ir frames to a raw .h264 file in arrival order and probe it. Record the actual resolution, the observed keyframe cadence and the achieved bitrate, and confirm whether framing is Annex-B with parameter sets on IDR frames. This is the step that resolves the parameters section 7.6 leaves undetermined, and it has to be repeated per machine class, because the transform selected differs between rigs with and without a usable hardware encoder. Note the backend from the H264 encoder: backend={Backend} {W}x{H}@{Fps} line alongside the result.
8. Confirm the encode size in that log line against the camera’s native size, and expect the declared width and height to be rounded up to a multiple of 16.
9. Disconnect the tracker while capturing. Confirm a warning containing Tobii device connection lost and attempting reconnect appears, and confirm that gaze frames keep publishing the last values with no staleness indicator. Reconnect and confirm values resume changing without restarting the capture session.
10. Stop and restart capture within the same host process. Confirm initialisation runs again, and confirm that the first frames of the second session carry the previous session’s retained values before the first callback of the new session arrives.
11. Start with CaptureVideoFeed off and confirm no camera.ir stream is published and no encode thread starts, while gaze, pupillometry and head pose publish normally.

15. Revision history

HEAT Capture team. Last updated 2 October 2026.