Visual aids tour

Visual aids are the layer above ImGui that makes a running app self-narrating for tutorials, demos, and recordings. Five overlays draw on top of any ImGui frame:

  • Highlight — flash a colored rectangle around a widget’s bbox for N frames. Pinpoints “this thing right here.”

  • Mouse trail — fading line behind io.MousePos, so the synth cursor’s path through the UI is visible in a recording.

  • Cursor sprite — in-frame pointer drawn at io.MousePos. Without it, recordings show no cursor (the OS-level cursor doesn’t reach the framebuffer).

  • Narrate — sticky-note callout with an optional connector line to a target widget. Auto-fits to avoid sibling-widget overlap.

  • Auto-highlight on command — global flag that fires highlight on every accepted live command’s target. One-shot debug aid for figuring out which widget a curl invocation actually hit.

Plus two more for keyboard work — exercised by this tutorial’s recording:

  • Key HUD — bottom-center keycap pops for every synth key event, plus a Ctrl/Shift/Alt/Super modifier strip lit while a modifier is held.

  • Focus rect — colored rectangle around the widget that owns io.active_widget. Shows where typing lands.

This is the richer reference for the recording surface — the recording tutorial walked through the driver script anatomy; this one walks through what each visual aid does in isolation.

Source: examples/tutorial/visual_aids_tour.das.

Walkthrough

The recording is voiced and self-verifying: each beat speaks a line while a real gesture fires under it and is asserted — clicking the highlight and narrate buttons (the click must register), scrubbing VOLUME (the value must change), and typing into NAME_INPUT then clearing it with Ctrl+A / Backspace (the buffer must end empty). A silently broken beat aborts the recording at teardown instead of shipping.

  1options gen2
  2
  3require imgui
  4require imgui_app
  5require opengl/opengl_boost
  6require live/glfw_live
  7require live/live_api
  8require live/live_commands
  9require live/live_vars
 10require live_host
 11require imgui/imgui_live
 12require imgui/imgui_boost_runtime
 13require imgui/imgui_boost_v2
 14require imgui/imgui_widgets_builtin
 15require imgui/imgui_containers_builtin
 16require imgui/imgui_visual_aids
 17
 18// =============================================================================
 19// TUTORIAL: visual_aids_tour — the keeper reference example for Phase 4/5.
 20//
 21// Main window holds four widgets being demonstrated:
 22//   - STATUS    : a text_show display (label-style)
 23//   - VOLUME    : a slider_float (interactive input)
 24//   - SAVE_BTN  : a button (trigger)
 25//   - NAME_INPUT: an input_text (typing target for the keyboard tour)
 26//
 27// Side "Controls" window has buttons that drive the visual aids:
 28//   - Highlight each demoed widget
 29//   - Toggle the mouse trail
 30//   - Pop a narrate callout pointing at VOLUME / SAVE_BTN
 31//   - Toggle auto-highlight-on-command (highlights every remote action)
 32//
 33// DRIVE (also exposed via curl on localhost:9090):
 34//   curl -X POST -d '{"name":"imgui_highlight","args":{"target":"VOLUME"}}' localhost:9090/command
 35//   curl -X POST -d '{"name":"imgui_mouse_trail","args":{"enabled":true}}' localhost:9090/command
 36//   curl -X POST -d '{"name":"imgui_narrate","args":{"text":"click me","target":"SAVE_BTN","frames":180}}' localhost:9090/command
 37//   curl -X POST -d '{"name":"screenshot","args":{"file":"visual_aids_tour.png"}}' localhost:9090/command
 38//
 39// STANDALONE: daslang.exe modules/dasImgui/examples/tutorial/visual_aids_tour.das
 40// LIVE:       daslang-live modules/dasImgui/examples/tutorial/visual_aids_tour.das
 41// =============================================================================
 42
 43[export]
 44def init() {
 45    live_create_window("visual aids tour", 1000, 640)
 46    live_imgui_init(live_window)
 47    DisableIniPersistence()
 48    let io & = unsafe(GetIO())
 49    GetStyle().FontScaleMain = 1.5
 50}
 51
 52[export]
 53def update() {
 54    if (!live_begin_frame()) return
 55    begin_frame()
 56
 57    ImGui_ImplGlfw_NewFrame()
 58    apply_synth_io_override()
 59    NewFrame()
 60
 61    // ===== Subject window — the demoed widgets =====
 62    SetNextWindowPos(float2(40.0f, 40.0f), ImGuiCond.FirstUseEver)
 63    SetNextWindowSize(float2(440.0f, 360.0f), ImGuiCond.FirstUseEver)
 64    window(SUBJECT_WIN, (text = "Demoed widgets", closable = false,
 65                         flags = ImGuiWindowFlags.None)) {
 66        text("These widgets get highlighted, narrated, typed-into, etc.")
 67        separator(VA_SEP_1)
 68        text_show(STATUS, (value = "ready"))
 69        VOLUME.bounds = (0.0f, 100.0f)
 70        slider_float(VOLUME, (text = "Volume"))
 71        if (button(SAVE_BTN, (text = "Save"))) {
 72            print("save clicked, count={SAVE_BTN.click_count}\n")
 73        }
 74        separator(VA_SEP_2)
 75        input_text(NAME_INPUT, (text = "Name"))
 76        text("buffer = {NAME_INPUT.value}")
 77    }
 78
 79    // ===== Controls window — driver buttons for the aids =====
 80    SetNextWindowPos(float2(520.0f, 40.0f), ImGuiCond.FirstUseEver)
 81    SetNextWindowSize(float2(440.0f, 560.0f), ImGuiCond.FirstUseEver)
 82    window(CONTROLS_WIN, (text = "Visual aids controls", closable = false,
 83                          flags = ImGuiWindowFlags.None)) {
 84        text("Highlight a widget (yellow rect, 60 frames):")
 85        separator(VA_SEP_3)
 86        if (button(BTN_HIGHLIGHT_STATUS, (text = "highlight STATUS"))) {
 87            highlight("SUBJECT_WIN/STATUS")
 88        }
 89        if (button(BTN_HIGHLIGHT_VOLUME, (text = "highlight VOLUME"))) {
 90            highlight("SUBJECT_WIN/VOLUME")
 91        }
 92        if (button(BTN_HIGHLIGHT_SAVE, (text = "highlight SAVE_BTN"))) {
 93            highlight("SUBJECT_WIN/SAVE_BTN")
 94        }
 95        if (button(BTN_HIGHLIGHT_ALL, (text = "highlight all three"))) {
 96            highlight("SUBJECT_WIN/STATUS")
 97            highlight("SUBJECT_WIN/VOLUME")
 98            highlight("SUBJECT_WIN/SAVE_BTN")
 99        }
100
101        spacing(VA_SP_1)
102        text("Mouse trail (fading dots following cursor):")
103        separator(VA_SEP_4)
104        if (button(BTN_TRAIL_ON, (text = "mouse trail ON"))) {
105            mouse_trail(true)
106        }
107        if (button(BTN_TRAIL_OFF, (text = "mouse trail OFF"))) {
108            mouse_trail(false)
109        }
110
111        spacing(VA_SP_2)
112        text("Narrate callout (overlay box + connector line):")
113        separator(VA_SEP_5)
114        if (button(BTN_NARRATE_VOLUME, (text = "narrate VOLUME"))) {
115            narrate("Drag this to set volume.", "SUBJECT_WIN/VOLUME")
116        }
117        if (button(BTN_NARRATE_SAVE, (text = "narrate SAVE_BTN"))) {
118            narrate("Click here to save.", "SUBJECT_WIN/SAVE_BTN")
119        }
120        if (button(BTN_NARRATE_FLOATING, (text = "narrate floating"))) {
121            narrate("Floating overlay, no target widget.")
122        }
123
124        spacing(VA_SP_3)
125        text("Auto-highlight on command (flag, currently {imgui_auto_highlight_on_command}):")
126        separator(VA_SEP_6)
127        if (button(BTN_AUTO_ON, (text = "auto-highlight ON"))) {
128            imgui_auto_highlight_on_command = true
129        }
130        if (button(BTN_AUTO_OFF, (text = "auto-highlight OFF"))) {
131            imgui_auto_highlight_on_command = false
132        }
133    }
134
135    end_of_frame()
136    Render()
137    var w, h : int
138    live_get_framebuffer_size(w, h)
139    glViewport(0, 0, w, h)
140    glClearColor(0.10f, 0.10f, 0.12f, 1.0f)
141    glClear(GL_COLOR_BUFFER_BIT)
142    live_imgui_render()
143
144    live_end_frame()
145}
146
147[export]
148def shutdown() {
149    live_imgui_shutdown()
150    live_destroy_window()
151}
152
153[export]
154def main() {
155    init()
156    while (!exit_requested()) {
157        update()
158    }
159    shutdown()
160}

Layout

The subject has two window(...) containers:

  • SUBJECT_WIN (titled “Demoed widgets”) — the things the visual aids point at: STATUS (text_show), VOLUME (slider_float), SAVE_BTN (button), NAME_INPUT (input_text). Leaves register at SUBJECT_WIN/<ident>.

  • CONTROLS_WIN (titled “Visual aids controls”) — buttons that fire each aid in-process so you can iterate without a separate driver shell.

In the recording at the top, the driver drives the mouse aids through the controls window — it clicks BTN_HIGHLIGHT_* and BTN_NARRATE_* for real (each click verified), so the buttons’ own handlers fire highlight / narrate. The keyboard overlays (imgui_key_hud / imgui_focus_rect) and the typing/chords are posted as direct live commands. The buttons exist so a user dropped into daslang-live can drive every aid by hand for exploration too.

Highlight

A colored rectangle drawn around a widget’s bbox for N frames:

highlight("SUBJECT_WIN/VOLUME")              // default: yellow, 60 frames
highlight("SUBJECT_WIN/SAVE_BTN", 120, 0xFFFF8030u)   // orange, 120 frames

Highlights are short by design — long enough for “look here” to register, short enough that two consecutive highlights compose visually rather than queue. The recording clicks the three highlight buttons in turn (status, volume, save), each flashing a rect on the matching widget. Tunable defaults live in imgui_visual_aids.das (highlight_default_frames, highlight_color).

Mouse trail

A fading line behind io.MousePos, drawn every frame the trail is enabled:

mouse_trail(true)                // on
mouse_trail(true, 0.45f)         // 450ms fade
mouse_trail(false)               // off

The trail’s value is mostly to make synth-cursor recordings parseable — without it, the cursor teleports between waypoints and the viewer can’t tell what happened. Real mouse motion shows a trail too, which is sometimes a nice touch in live demos.

Cursor sprite

A visible mouse-pointer sprite drawn at io.MousePos:

cursor_sprite(true)
cursor_sprite(false)

OS-level cursors don’t reach the framebuffer — the screen recorder sees ImGui’s draw output, not the cursor that the WM compositor draws on top. Without cursor_sprite, a recording shows widgets reacting to clicks with no visible “thing” doing the clicking. Always enable this before record_start for any tutorial recording.

Narrate

A sticky-note callout with optional connector line to a target widget:

narrate("Click here to save.", "SUBJECT_WIN/SAVE_BTN")   // 180 frames default
narrate("Drag this to set volume.", "SUBJECT_WIN/VOLUME", 240)
narrate("Floating overlay, no target.")                   // no connector line

The auto-fit logic tries four candidate anchors (right / left / below / above the target widget) and picks the first that doesn’t overlap the widget OR the viewport edge OR (when enabled) the key_hud zone. Falls back to right-then-clamp only when every candidate overflows. The result: a sticky-note that points at the right thing without covering it.

Auto-highlight on command

A module-scope flag that fires highlight on every accepted live command’s target:

imgui_auto_highlight_on_command = true
// any imgui_click / imgui_force_set / imgui_open ... now flashes its target

Useful when debugging “why didn’t my curl do anything” — turn it on, fire the command, and either see a highlight flash (command reached the right widget) or no flash (typo in the target path, or the widget isn’t in the registry that frame).

Key HUD + focus rect (recording’s keyboard tour)

Beyond the mouse-focused aids:

  • imgui_key_hud pops a keycap label at bottom-center for every synthesized key event. imgui_focus_rect draws a colored rectangle around whichever widget has keyboard focus right now.

  • The recording’s second half exercises both: a real click on NAME_INPUT lights the focus rect, then imgui_key_type types “Hello, World!” into the input — every keycap pops at the bottom with the matching mod-strip flash on H, W, and ! (auto-shift keys). The committed value is verified. Then imgui_key_chord fires Ctrl+A — Ctrl pill lights up while the “A” keycap pops — and a Backspace clears the (selected) buffer, verified empty.

  • All three keyboard live commands route through the L1 synth IO layer described in Driving from outside.

Standalone vs live

The visual-aid functions (highlight, mouse_trail, narrate, cursor_sprite) work in both modes — they’re just drawing code. The live-command wrappers (imgui_highlight / imgui_mouse_trail / etc.) need daslang-live for the HTTP surface.

Driving from outside

Every aid is reachable via curl:

curl -X POST -d '{"name":"imgui_highlight","args":{"target":"SUBJECT_WIN/VOLUME"}}' \
     localhost:9090/command
curl -X POST -d '{"name":"imgui_mouse_trail","args":{"enabled":true}}' \
     localhost:9090/command
curl -X POST -d '{"name":"imgui_cursor_sprite","args":{"enabled":true}}' \
     localhost:9090/command
curl -X POST -d '{"name":"imgui_narrate","args":{"text":"click me","target":"SUBJECT_WIN/SAVE_BTN","frames":180}}' \
     localhost:9090/command
curl -X POST -d '{"name":"imgui_auto_highlight","args":{"enabled":true}}' \
     localhost:9090/command
curl -X POST -d '{"name":"imgui_key_hud","args":{"enabled":true,"show_modifiers":true}}' \
     localhost:9090/command
curl -X POST -d '{"name":"imgui_focus_rect","args":{"enabled":true}}' \
     localhost:9090/command

The recording at the top of this page fires this exact sequence (plus the keyboard tour) through the playwright transport instead of curl.

See also

Full source: examples/tutorial/visual_aids_tour.das

Driver: tests/integration/record_visual_aids_tour.das — the voiced, self-verifying tour that walks every aid in turn.

Implementation: modules/dasImgui/widgets/imgui_visual_aids.das — the full surface plus narrate auto-fit, key HUD, focus rect.

Previous tutorial: Driving from outside

Next tutorial: Recording tutorial videos

Boost macros — the macro layer.