Visual aids tour

Visual aids are the layer above ImGui that makes a running app self-narrating for tutorials, demos, and recordings. Five overlays draw on top of any ImGui frame:

  • Highlight — flash a colored rectangle around a widget’s bbox for N frames. Pinpoints “this thing right here.”

  • Mouse trail — fading line behind io.MousePos, so the synth cursor’s path through the UI is visible in a recording.

  • Cursor sprite — in-frame pointer drawn at io.MousePos. Without it, recordings show no cursor (the OS-level cursor doesn’t reach the framebuffer).

  • Narrate — sticky-note callout with an optional connector line to a target widget. Auto-fits to avoid sibling-widget overlap.

  • Auto-highlight on command — global flag that fires highlight on every accepted live command’s target. One-shot debug aid for figuring out which widget a curl invocation actually hit.

Plus two more for keyboard work — exercised by this tutorial’s recording:

  • Key HUD — bottom-center keycap pops for every synth key event, plus a Ctrl/Shift/Alt/Super modifier strip lit while a modifier is held.

  • Focus rect — colored rectangle around the widget that owns io.active_widget. Shows where typing lands.

This is the richer reference for the recording surface — the recording tutorial walked through the driver script anatomy; this one walks through what each visual aid does in isolation.

Source: examples/tutorial/visual_aids_tour.das.

Walkthrough

visual_aids_tour recording 
  1options gen2
  2
  3require imgui
  4require imgui_app
  5require glfw/glfw_boost
  6require opengl/opengl_boost
  7require live/glfw_live
  8require live/live_api
  9require live/live_commands
 10require live/live_vars
 11require live/opengl_live
 12require live_host
 13require imgui/imgui_live
 14require imgui/imgui_boost_runtime
 15require imgui/imgui_boost_v2
 16require imgui/imgui_widgets_builtin
 17require imgui/imgui_containers_builtin
 18require imgui/imgui_visual_aids
 19
 20// =============================================================================
 21// TUTORIAL: visual_aids_tour — the keeper reference example for Phase 4/5.
 22//
 23// Main window holds four widgets being demonstrated:
 24//   - STATUS    : a text_show display (label-style)
 25//   - VOLUME    : a slider_float (interactive input)
 26//   - SAVE_BTN  : a button (trigger)
 27//   - NAME_INPUT: an input_text (typing target for the keyboard tour)
 28//
 29// Side "Controls" window has buttons that drive the visual aids:
 30//   - Highlight each demoed widget
 31//   - Toggle the mouse trail
 32//   - Pop a narrate callout pointing at VOLUME / SAVE_BTN
 33//   - Toggle auto-highlight-on-command (highlights every remote action)
 34//
 35// DRIVE (also exposed via curl on localhost:9090):
 36//   curl -X POST -d '{"name":"imgui_highlight","args":{"target":"VOLUME"}}' localhost:9090/command
 37//   curl -X POST -d '{"name":"imgui_mouse_trail","args":{"enabled":true}}' localhost:9090/command
 38//   curl -X POST -d '{"name":"imgui_narrate","args":{"text":"click me","target":"SAVE_BTN","frames":180}}' localhost:9090/command
 39//   curl -X POST -d '{"name":"screenshot","args":{"file":"visual_aids_tour.png"}}' localhost:9090/command
 40//
 41// STANDALONE: daslang.exe modules/dasImgui/examples/tutorial/visual_aids_tour.das
 42// LIVE:       daslang-live modules/dasImgui/examples/tutorial/visual_aids_tour.das
 43// =============================================================================
 44
 45[export]
 46def init() {
 47    live_create_window("Phase 4 — visual aids tour", 1000, 640)
 48    live_imgui_init(live_window)
 49    var io & = unsafe(GetIO())
 50    io.FontGlobalScale = 1.5
 51}
 52
 53[export]
 54def update() {
 55    if (!live_begin_frame()) return
 56    begin_frame()
 57
 58    ImGui_ImplOpenGL3_NewFrame()
 59    ImGui_ImplGlfw_NewFrame()
 60    apply_synth_io_override()
 61    NewFrame()
 62
 63    // ===== Subject window — the demoed widgets =====
 64    SetNextWindowPos(float2(40.0f, 40.0f), ImGuiCond.FirstUseEver)
 65    SetNextWindowSize(float2(440.0f, 360.0f), ImGuiCond.FirstUseEver)
 66    window(SUBJECT_WIN, (text = "Demoed widgets", closable = false,
 67                         flags = ImGuiWindowFlags.None)) {
 68        text("These widgets get highlighted, narrated, typed-into, etc.")
 69        separator(VA_SEP_1)
 70        text_show(STATUS, (value = "ready"))
 71        VOLUME.bounds = (0.0f, 100.0f)
 72        slider_float(VOLUME, (text = "Volume"))
 73        if (button(SAVE_BTN, (text = "Save"))) {
 74            print("save clicked, count={SAVE_BTN.click_count}\n")
 75        }
 76        separator(VA_SEP_2)
 77        input_text(NAME_INPUT, (text = "Name"))
 78        text("buffer = {NAME_INPUT.value}")
 79    }
 80
 81    // ===== Controls window — driver buttons for the aids =====
 82    SetNextWindowPos(float2(520.0f, 40.0f), ImGuiCond.FirstUseEver)
 83    SetNextWindowSize(float2(440.0f, 560.0f), ImGuiCond.FirstUseEver)
 84    window(CONTROLS_WIN, (text = "Visual aids controls", closable = false,
 85                          flags = ImGuiWindowFlags.None)) {
 86        text("Highlight a widget (yellow rect, 60 frames):")
 87        separator(VA_SEP_3)
 88        if (button(BTN_HIGHLIGHT_STATUS, (text = "highlight STATUS"))) {
 89            highlight("SUBJECT_WIN/STATUS")
 90        }
 91        if (button(BTN_HIGHLIGHT_VOLUME, (text = "highlight VOLUME"))) {
 92            highlight("SUBJECT_WIN/VOLUME")
 93        }
 94        if (button(BTN_HIGHLIGHT_SAVE, (text = "highlight SAVE_BTN"))) {
 95            highlight("SUBJECT_WIN/SAVE_BTN")
 96        }
 97        if (button(BTN_HIGHLIGHT_ALL, (text = "highlight all three"))) {
 98            highlight("SUBJECT_WIN/STATUS")
 99            highlight("SUBJECT_WIN/VOLUME")
100            highlight("SUBJECT_WIN/SAVE_BTN")
101        }
102
103        spacing(VA_SP_1)
104        text("Mouse trail (fading dots following cursor):")
105        separator(VA_SEP_4)
106        if (button(BTN_TRAIL_ON, (text = "mouse trail ON"))) {
107            mouse_trail(true)
108        }
109        if (button(BTN_TRAIL_OFF, (text = "mouse trail OFF"))) {
110            mouse_trail(false)
111        }
112
113        spacing(VA_SP_2)
114        text("Narrate callout (overlay box + connector line):")
115        separator(VA_SEP_5)
116        if (button(BTN_NARRATE_VOLUME, (text = "narrate VOLUME"))) {
117            narrate("Drag this to set volume.", "SUBJECT_WIN/VOLUME")
118        }
119        if (button(BTN_NARRATE_SAVE, (text = "narrate SAVE_BTN"))) {
120            narrate("Click here to save.", "SUBJECT_WIN/SAVE_BTN")
121        }
122        if (button(BTN_NARRATE_FLOATING, (text = "narrate floating"))) {
123            narrate("Floating overlay, no target widget.")
124        }
125
126        spacing(VA_SP_3)
127        text("Auto-highlight on command (flag, currently {imgui_auto_highlight_on_command}):")
128        separator(VA_SEP_6)
129        if (button(BTN_AUTO_ON, (text = "auto-highlight ON"))) {
130            imgui_auto_highlight_on_command = true
131        }
132        if (button(BTN_AUTO_OFF, (text = "auto-highlight OFF"))) {
133            imgui_auto_highlight_on_command = false
134        }
135    }
136
137    end_of_frame()
138    Render()
139    var w, h : int
140    live_get_framebuffer_size(w, h)
141    glViewport(0, 0, w, h)
142    glClearColor(0.10f, 0.10f, 0.12f, 1.0f)
143    glClear(GL_COLOR_BUFFER_BIT)
144    ImGui_ImplOpenGL3_RenderDrawData(GetDrawData())
145
146    live_end_frame()
147}
148
149[export]
150def shutdown() {
151    live_imgui_shutdown()
152    live_destroy_window()
153}
154
155[export]
156def main() {
157    init()
158    while (!exit_requested()) {
159        update()
160    }
161    shutdown()
162}

Layout

The subject has two window(...) containers:

  • SUBJECT_WIN (titled “Demoed widgets”) — the things the visual aids point at: STATUS (text_show), VOLUME (slider_float), SAVE_BTN (button), NAME_INPUT (input_text). Leaves register at SUBJECT_WIN/<ident>.

  • CONTROLS_WIN (titled “Visual aids controls”) — buttons that fire each aid in-process so you can iterate without a separate driver shell.

In the recording at the top, the driver bypasses the controls window entirely — it calls imgui_highlight / imgui_mouse_trail / imgui_narrate / imgui_focus / imgui_key_type directly via live commands. The buttons exist so a user dropped into daslang-live can drive every aid by hand for exploration.

Highlight

A colored rectangle drawn around a widget’s bbox for N frames:

highlight("SUBJECT_WIN/VOLUME")              // default: yellow, 60 frames
highlight("SUBJECT_WIN/SAVE_BTN", 120, 0xFFFF8030u)   // orange, 120 frames

Highlights are short by design — long enough for “look here” to register, short enough that two consecutive highlights compose visually rather than queue. The recording fires three highlights in ~2 seconds; the third lands while the first is still fading. Tunable defaults live in imgui_visual_aids.das (highlight_default_frames, highlight_color).

Mouse trail

A fading line behind io.MousePos, drawn every frame the trail is enabled:

mouse_trail(true)                // on
mouse_trail(true, 0.45f)         // 450ms fade
mouse_trail(false)               // off

The trail’s value is mostly to make synth-cursor recordings parseable — without it, the cursor teleports between waypoints and the viewer can’t tell what happened. Real mouse motion shows a trail too, which is sometimes a nice touch in live demos.

Cursor sprite

A visible mouse-pointer sprite drawn at io.MousePos:

cursor_sprite(true)
cursor_sprite(false)

OS-level cursors don’t reach the framebuffer — the screen recorder sees ImGui’s draw output, not the cursor that the WM compositor draws on top. Without cursor_sprite, a recording shows widgets reacting to clicks with no visible “thing” doing the clicking. Always enable this before record_start for any tutorial recording.

Narrate

A sticky-note callout with optional connector line to a target widget:

narrate("Click here to save.", "SUBJECT_WIN/SAVE_BTN")   // 180 frames default
narrate("Drag this to set volume.", "SUBJECT_WIN/VOLUME", 240)
narrate("Floating overlay, no target.")                   // no connector line

The auto-fit logic tries four candidate anchors (right / left / below / above the target widget) and picks the first that doesn’t overlap the widget OR the viewport edge OR (when enabled) the key_hud zone. Falls back to right-then-clamp only when every candidate overflows. The result: a sticky-note that points at the right thing without covering it.

Auto-highlight on command

A module-scope flag that fires highlight on every accepted live command’s target:

imgui_auto_highlight_on_command = true
// any imgui_click / imgui_set / imgui_open ... now flashes its target

Useful when debugging “why didn’t my curl do anything” — turn it on, fire the command, and either see a highlight flash (command reached the right widget) or no flash (typo in the target path, or the widget isn’t in the registry that frame).

Key HUD + focus rect (recording’s keyboard tour)

Beyond the mouse-focused aids:

  • imgui_key_hud pops a keycap label at bottom-center for every synthesized key event. imgui_focus_rect draws a colored rectangle around whichever widget has keyboard focus right now.

  • The recording’s second half exercises both: imgui_focus on NAME_INPUT lights the focus rect, then imgui_key_type types “Hello, World!” into the input — every keycap pops at the bottom with the matching mod-strip flash on H, W, and ! (auto-shift keys). Then imgui_key_chord fires Ctrl+A — Ctrl pill lights up while the “A” keycap pops. Finally a Backspace clears the (selected) buffer.

  • All three keyboard live commands route through the L1 synth IO layer described in Driving from outside.

Standalone vs live

The visual-aid functions (highlight, mouse_trail, narrate, cursor_sprite) work in both modes — they’re just drawing code. The live-command wrappers (imgui_highlight / imgui_mouse_trail / etc.) need daslang-live for the HTTP surface.

Driving from outside

Every aid is reachable via curl:

curl -X POST -d '{"name":"imgui_highlight","args":{"target":"SUBJECT_WIN/VOLUME"}}' \
     localhost:9090/command
curl -X POST -d '{"name":"imgui_mouse_trail","args":{"enabled":true}}' \
     localhost:9090/command
curl -X POST -d '{"name":"imgui_cursor_sprite","args":{"enabled":true}}' \
     localhost:9090/command
curl -X POST -d '{"name":"imgui_narrate","args":{"text":"click me","target":"SUBJECT_WIN/SAVE_BTN","frames":180}}' \
     localhost:9090/command
curl -X POST -d '{"name":"imgui_auto_highlight","args":{"enabled":true}}' \
     localhost:9090/command
curl -X POST -d '{"name":"imgui_key_hud","args":{"enabled":true,"show_modifiers":true}}' \
     localhost:9090/command
curl -X POST -d '{"name":"imgui_focus_rect","args":{"enabled":true}}' \
     localhost:9090/command

The recording at the top of this page fires this exact sequence (plus the keyboard tour) through the playwright transport instead of curl.

See also

Full source: examples/tutorial/visual_aids_tour.das

Driver: tests/integration/record_visual_aids.das — the Phase 4/5 keeper, walks every aid through a 30-second tour.

Implementation: modules/dasImgui/widgets/imgui_visual_aids.das — the full surface plus narrate auto-fit, key HUD, focus rect.

Previous tutorial: Driving from outside

Next tutorial: Recording APNGs

Boost macros — the macro layer.