Interface control document

1. Identification

2. Scope

This document defines the audio interfaces the module opens against the local Windows audio stack, the local Whisper inference dependency and its model prerequisite, the utterance state machine that drives publication, and the events the module publishes to HEAT Capture.

The module is receive-only with respect to audio. It never renders audio, never writes to a device, and exposes no remote control endpoint. Its only outbound interfaces are frame publication through the Module SDK and the debug view.

3. Interface inventory

4. IF-CMMIC-01 Peak-level metering

This is the only interface the module opens with default settings. It is what decides whether speech is active; it is not published in any form.

4.1 Transport and addressing

Device selection happens once, during initialisation. Changing DeviceId or CaptureMode while the module is initialised does not re-resolve the endpoint; the host must reinitialise the module.

4.2 Wire format and conversion

The linear peak is converted to dBFS as 20 * log10(peak). A peak of exactly 0 yields negative infinity rather than a finite floor. The activity decision is:

peak    = max(micPeakDbfs, desktopPeakDbfs)
isActive = peak > SilenceThresholdDb

desktopPeakDbfs is negative infinity unless MicPlusDesktopSeparated is active, so in MicOnly the comparison is against the microphone alone. In MicPlusDesktopSeparated the louder of the two endpoints wins, which means desktop output on its own can open and close an utterance with the microphone silent.

The negative-infinity case is substituted with -80 only when storing the value for the debug view. The activity comparison uses the unsubstituted value.

4.3 Authentication

None. The module reads whatever endpoints the logged-in Windows session exposes, with the permissions of the HEAT Capture process.

4.4 Timing

Metering is polled at the SDK Interval, 1000 ms by default. Peak level is an instantaneous sample at the moment of the poll, not an average over the interval, so speech quieter than SilenceThresholdDb at that instant is not detected at all, and a transient between polls is invisible.

5. IF-CMMIC-02 Microphone capture stream

5.1 When the stream is opened

A capture stream is opened at capture start only when at least one of EnableAudioInComms, EnableTranscription, or CaptureMode = MicPlusDesktopSeparated is set. With defaults, no audio stream is ever opened and no audio leaves the endpoint.

5.2 Format

The module requests no format and no share mode. It constructs WasapiCapture with the endpoint alone and then adopts the WaveFormat that NAudio reports for that stream as the authority for sample rate, bit depth and channel count. The module never resamples, requantises or downmixes before publication: buffered bytes are exactly the bytes the WASAPI callback delivered.

Because the format is not fixed, a consumer must read the format parameters from each published audio event rather than assume a rate or depth. There is no setting that constrains the format.

5.3 Exclusive versus shared mode

The module never requests exclusive mode. If another application already holds the endpoint in exclusive mode, constructing the capture throws COMException with HRESULT 0x88890004 (AUDCLNT_E_DEVICE_IN_USE). That case is caught specifically: the module logs a warning containing Microphone in exclusive use and capture disabled, monitoring only, sets its internal exclusive-mode flag, and disposes the capture. Metering (IF-CMMIC-01) is unaffected, so utterance start and end events continue to be published with no audio and no transcript behind them.

The exclusive-mode flag is the only source for the secure field on a transcript event. Since exclusive mode also means no microphone audio was buffered, no transcript can be produced in that state, so secure: true is unreachable in this revision.

5.4 Buffering gate

The capture callback appends to an in-memory buffer only while the module considers speech active. Callbacks that arrive while inactive are discarded at the callback, not filtered later. The buffer is unbounded in the module; the only size signal is the debug view gauge, whose scale ends at 2048 KB.

5.5 Lifecycle and errors

Capture start opens the stream and logs “Microphone capture started (buffering for comms events)”. Any exception other than the exclusive-mode case logs “Failed to start audio capture” and disposes the capture, leaving metering-only operation; capture start still reports success to the host.

If the stream stops on its own with an exception, the module logs “Microphone capture stopped unexpectedly” and does nothing else. There is no reconnect, no retry and no backoff on any audio interface in this revision: a device removed or reset mid-session stays down for the remainder of the session, while metering failures are silent (see section 11).

Capture end disposes both streams, clears both buffers, and resets the active, exclusive-mode, buffering and silence-timer state.

6. IF-CMMIC-03 Desktop loopback capture stream

Opened only when CaptureMode is MicPlusDesktopSeparated, compared case-insensitively after trimming. Any other value, including an empty one, means microphone only.

The loopback stream contributes to the activity decision through IF-CMMIC-01 and is published as its own audio event, but it is never transcribed. Transcription runs against microphone bytes only.

7. IF-CMMIC-04 Audio session process attribution

When a cycle is active, the module enumerates audio sessions to label the utterance with a process name. In MicPlusDesktopSeparated it queries the render endpoint first and falls back to the capture endpoint; in MicOnly it queries the capture endpoint only.

Selection rule, per endpoint: sessions that are not in the active state or report process id 0 are skipped; the first active session belonging to a process other than HEAT Capture wins; if the only active session is HEAT Capture’s own process, that name is used; otherwise the label is the empty string. Resolving a process id that has already exited is swallowed and the session skipped.

The label is refreshed only on cycles where audio is active, and the value in effect when the utterance closes is the one attached to every event of that utterance. It is a best-effort attribution, not evidence of which application received the speech.

Enumeration failure leaves the previous label in place and, with DebugLogs on, logs “Failed to enumerate audio sessions”.

8. IF-CMMIC-05 Transcription runtime and model

8.1 Model prerequisite and provenance

Transcription is in-process. There is no service, socket or child process, and nothing leaves the workstation.

8.2 Native runtime selection

Before loading, the module points the Whisper.net native loader at its own assembly location, because the packaged layout places Whisper.net.dll in a subdirectory while the native runtimes tree sits at the package root. The runtime preference order is fixed in code: CUDA, Vulkan, CoreML, CPU, then CPU without AVX. It is not configurable, and the module does not log which runtime was selected.

8.3 Load lifecycle

The model is loaded once, during module initialisation, and only when EnableTranscription is set. Success logs Whisper model loaded: {Path}. Any load failure logs “Failed to load Whisper model” and fails initialisation.

The module does not raise the SDK StartupTimeout, so model load, native library probing and runtime selection must all complete inside the default 5 second startup budget. A cold first load of a 142 MB model, or GPU runtime probing on a machine without a usable GPU, can exceed it.

Capture end disposes the Whisper processor and factory, while the model is loaded only during initialisation, which the host calls once per module lifetime. Transcription therefore works for the first capture session after initialisation and is silently absent from later sessions in the same host process, because the publication path is gated on a live processor rather than on the setting. See section 11.

8.4 Conversion and invocation

For each completed utterance the buffered microphone PCM is converted before inference: downmixed to mono when the capture format has more than one channel, resampled to 16000 Hz, requantised to 16-bit PCM, and wrapped in a WAV container in memory. Segment texts returned by the processor are concatenated with no separator and the result is trimmed.

Language is passed straight through to the processor, default auto for automatic detection. It is not validated by the module.

Inference is awaited inline inside the capture cycle. HandleCapture has no timeout, so a slow transcription delays the next cycle and therefore delays the next activity poll for as long as it runs.

9. IF-CMMIC-06 Frame publication and data model reference

9.1 Utterance state machine

Every branch below is evaluated once per capture cycle, inside HandleCapture.

A cycle that matches none of the three returns nothing, and the SDK publishes nothing for that cycle.

Two consequences follow from evaluating this only at the capture cadence. First, the silence gap is measured in whole cycles: with the default Interval of 1000 ms and the default SilenceGapMs of 500 ms, the gap is already satisfied at the next cycle, so an utterance closes on the first quiet poll and any SilenceGapMs below Interval behaves identically. Second, audio is only buffered between the poll that saw speech and the poll that did not, so the trailing part of an utterance after the last active poll is never captured.

Audit finding MIC-01 applies to this revision. Step 2 clears the active state on a single inactive poll and stops buffering before the configured gap has expired, and step 1 then treats resumed speech as a new utterance and clears the buffer. An intra-sentence pause therefore produces a second start event and discards the audio buffered for the first part of the sentence. Utterance events in captured data are not a reliable count of sentences spoken, and a published utterance is not guaranteed to hold all of the speech that preceded it.

9.2 Entity identity

Events carry an entity id derived from the capture endpoint’s WASAPI device id string: the string’s hash code, truncated to 32 bits and widened. The module publishes no entity and no component: a frame from this module contains only events. A consumer that expects a declared entity for that id will not find one from this module. The .NET runtime does not contract string hash codes as stable across processes, so the value should not be treated as a persistent identifier for the device across host restarts.

9.3 Published messages

Every message below is an EntityEvent on the frame. Frame capture time and every event timestamp in a cycle use the same value, UTC Unix milliseconds read at the top of that cycle.

COMMS_TYPE_TTS is the text-to-speech enum value, reused here for a speech-to-text transcript. The frequency and radio_band fields are never set by this module. secure is set only on the transcript event; audio events omit it regardless of exclusive mode.

When the capture format is unknown at close, the audio content header degrades to audio/pcm;src=<source>;enc=base64, with no format parameters, and the payload is published anyway.

Audio and transcript events are complementary, not alternatives: with both settings on, a single utterance close publishes the end event, one audio event per source, and the transcript event, all in the same frame.

9.4 SDK envelope

The SDK wraps the returned frame as a DataPacket with TimestampUnixMs set to UTC now at publication and the frame packed into Payload, carrying NodeId when OverrideNodeId is set and -1 otherwise. See the Module SDK ICD.

10. IF-CMMIC-07 Debug view

The module implements the SDK debug view contract, which the backend polls every 250 ms while the module is running and broadcasts to connected frontend clients. The state object exposes capture mode, voice-active and exclusive-mode indicators, the attributed process name, both buffer sizes in KB, both peak levels in dBFS rounded to one decimal, both capture formats as <rate>Hz / <bits>bit / <channels>ch, and the last transcript text.

This interface carries speech content: the last transcript and live level meters are broadcast to any client authorised to view the module’s debug state, whether or not transcripts are being published into captured data. Buffer sizes and peak levels are read from the capture loop’s own state, so the debug view is the only place the module exposes level or format information.

11. Error behaviour

Five of these paths are visible only when DebugLogs is enabled. With it off, lost audio bytes, failed transcriptions and failed peak reads leave no trace, so a session with missing utterances cannot be diagnosed after the fact unless DebugLogs was on while it ran.

12. Units and conventions

13. Configuration affecting the interfaces

Settings live under the HEAT.MicrophoneInput section.

The generic SDK settings apply as documented in Manage capture modules. Interval is the only one that changes interface behaviour: it sets the metering cadence and the effective resolution of the silence gap.

The module’s own README also documents a PublishRmsLevel setting and a MicrophoneLevel reading on a BiometricSensorComponent. Neither exists in this revision, and no such reading is published under any configuration.

14. Compatibility and versioning

There is no version negotiation on any interface. The module depends on WASAPI through NAudio 2.2.1, on the SDK 2.0.5 publication contract, and on Whisper.net 1.7.2 with a GGML model file whose format is defined by whisper.cpp. A GGML format change that Whisper.net 1.7.2 cannot read surfaces as a failed initialisation, not as a degraded session.

The audio format is not part of this contract. It is whatever the endpoint provides, so the same configuration on two workstations can publish different sample rates and bit depths. Consumers must read the format parameters from the event content.

The published message types are stable enum values in the shared schema. The reuse of COMMS_TYPE_TTS for transcripts is a property of this module, not of the schema, so a consumer filtering on COMMS_TYPE_VOICE alone will not see transcripts.

15. Security considerations

Audio and transcripts are personal data. This module can record what a person said and a machine-readable transcript of it, tagged with the process they were speaking to and, in MicPlusDesktopSeparated mode, the audio of everyone else on the call coming out of the speakers. Treat both as personal data subject to the deployment’s own data-protection obligations, and obtain consent from every participant, not only the workstation user, before enabling either setting.

The defaults are the privacy-preserving configuration: no stream is opened, and only voice-activity timing is published. Both EnableAudioInComms and EnableTranscription are policy decisions, not tuning.

16. Conformance

1. With default settings, start a capture session and speak. Confirm start and end tracking events appear, and confirm no audio event, no transcript and no level reading are published.
2. Confirm the log records the selected device name and id, and that it matches DeviceId when one is set.
3. Enable EnableAudioInComms and repeat. Confirm one voice CommsEvent per utterance, and record the rate, bits and ch values from its content as the effective format for that workstation.
4. Enable EnableTranscription and confirm the log records the loaded model path, then confirm a transcript CommsEvent with COMMS_TYPE_TTS for a spoken phrase.
5. Stop the session, start a new one without restarting HEAT Capture, and speak again. Confirm whether transcript events still appear. Their absence is the disposal behaviour described in 8.3.
6. Rename or remove the model file with transcription enabled and confirm initialisation fails with “Whisper model not found”, rather than starting without transcription.
7. Open the device in exclusive mode from another application and confirm the exclusive-use warning, that tracking events continue, and that no audio or transcript is published.
8. Speak a sentence containing a short pause. Count the start events. Until MIC-01 is fixed, expect more than one utterance for a single sentence and expect the audio of the part before the pause to be absent from the published payload.
9. Set SilenceGapMs below the SDK Interval and confirm utterance closing is unchanged, per the cadence limit in 9.1.
10. In MicPlusDesktopSeparated mode, play audio through the speakers with the microphone muted and confirm an utterance is opened and a src=desktop audio event is published.

17. Revision history

HEAT Capture team. Last updated 2 October 2026.