Driving from outside

Every boost widget that previous tutorials wrote registers a path in the telemetry tree (DRIVE_WIN/USER, DRIVE_WIN/SPEED, DRIVE_WIN/RUN_BTN …). That path is also an HTTP endpoint: the boost layer ships [live_command] handlers (imgui_snapshot, imgui_force_set, imgui_click, imgui_open, imgui_close, imgui_focus) that look up the target in the registry. imgui_click fires a real synthetic mouse click at the widget’s center; imgui_force_set / imgui_open / imgui_close write the matching field on the widget’s state struct directly. Either way ImGui sees the effect as if a real input device — or an external editor — drove it.

This tutorial flips the point of view: instead of writing the daslang side, write the driver — a curl / Python / Bash script that issues JSON commands at a running daslang-live app. Every interaction the user could perform via mouse/keyboard has a curl equivalent, and the two surfaces use one in-memory model.

Source: examples/tutorial/driving_outside.das — a small target app exposing five widget kinds. The recording is driven entirely by JSON commands: value writes and container toggles mutate state directly (no mouse), while imgui_click fires a real synthetic click.

Walkthrough

  1options gen2
  2
  3require imgui
  4require imgui_app
  5require opengl/opengl_boost
  6require live/glfw_live
  7require live/live_api
  8require live/live_commands
  9require live/live_vars
 10require live_host
 11require imgui/imgui_live
 12require imgui/imgui_boost_runtime
 13require imgui/imgui_boost_v2
 14require imgui/imgui_widgets_builtin
 15require imgui/imgui_containers_builtin
 16require imgui/imgui_visual_aids
 17
 18let PRESETS : array<string> <- ["Calm", "Mellow", "Active", "Frantic"]
 19var PRESET_ROW : table<int; ClickState>
 20
 21// =============================================================================
 22// TUTORIAL: driving_outside — every boost widget is also a JSON endpoint.
 23//
 24// Previous tutorials wrote the daslang side. This one is the inverse view:
 25// what the JSON command surface looks like as a programming model in its
 26// own right. Every widget the boost layer ships registers a path in the
 27// telemetry tree; that path is also addressable from outside via the
 28// [live_command] HTTP endpoints `imgui_snapshot` / `imgui_force_set` /
 29// `imgui_click` / `imgui_open` / `imgui_close` / `imgui_focus`.
 30//
 31// The target app is small — slider + button + input + popup + combo — so
 32// the driving recipes are easy to spot. Every interaction the user could
 33// perform via mouse/keyboard has a curl equivalent.
 34//
 35// STANDALONE: daslang.exe modules/dasImgui/examples/tutorial/driving_outside.das
 36// LIVE:       daslang-live modules/dasImgui/examples/tutorial/driving_outside.das
 37//
 38// DRIVE (curl recipes — pair these with the RST walkthrough):
 39//
 40//   # Snapshot the world (always the first read in any driver)
 41//   curl -X POST -d '{"name":"imgui_snapshot"}' localhost:9090/command
 42//
 43//   # imgui_force_set — drive a slider value
 44//   curl -X POST -d '{"name":"imgui_force_set","args":{"target":"DRIVE_WIN/SPEED","value":7}}' \
 45//        localhost:9090/command
 46//
 47//   # imgui_click — fire a button
 48//   curl -X POST -d '{"name":"imgui_click","args":{"target":"DRIVE_WIN/RUN_BTN"}}' \
 49//        localhost:9090/command
 50//
 51//   # imgui_force_set — string into a text input
 52//   curl -X POST -d '{"name":"imgui_force_set","args":{"target":"DRIVE_WIN/USER","value":"Alice"}}' \
 53//        localhost:9090/command
 54//
 55//   # imgui_force_set — combo by selected-index
 56//   curl -X POST -d '{"name":"imgui_force_set","args":{"target":"DRIVE_WIN/PRESET","value":2}}' \
 57//        localhost:9090/command
 58//
 59//   # imgui_open / imgui_close — popups, closable windows, tabs
 60//   curl -X POST -d '{"name":"imgui_open","args":{"target":"DRIVE_WIN/STATUS_POPUP"}}' \
 61//        localhost:9090/command
 62//   curl -X POST -d '{"name":"imgui_close","args":{"target":"DRIVE_WIN/STATUS_POPUP"}}' \
 63//        localhost:9090/command
 64// =============================================================================
 65
 66[export]
 67def init() {
 68    live_create_window("dasImgui driving_outside tutorial", 880, 620)
 69    live_imgui_init(live_window)
 70    // Deterministic FirstUseEver layout for the recording: disable imgui.ini so a prior
 71    // session's window pos/size can't drift the framing. Tutorial-scoped on purpose.
 72    DisableIniPersistence()
 73    let io & = unsafe(GetIO())
 74    GetStyle().FontScaleMain = 1.5
 75}
 76
 77[export]
 78def update() {
 79    if (!live_begin_frame()) return
 80    begin_frame()
 81
 82    ImGui_ImplGlfw_NewFrame()
 83    apply_synth_io_override()
 84    NewFrame()
 85
 86    SetNextWindowPos(ImVec2(30.0f, 30.0f), ImGuiCond.FirstUseEver)
 87    SetNextWindowSize(ImVec2(640.0f, 460.0f), ImGuiCond.FirstUseEver)
 88    window(DRIVE_WIN, (text = "driving_outside", closable = false,
 89                       flags = ImGuiWindowFlags.None)) {
 90
 91        // ---- A text input — driven by `imgui_force_set` with a string value ----
 92        input_text(USER, (text = "User"))
 93        text("USER.value = \"{USER.value}\"")
 94
 95        separator(DR_SEP_1)
 96
 97        // ---- A slider — driven by `imgui_force_set` with a number value ----
 98        slider_int(SPEED, (text = "Speed"))
 99        text("SPEED.value = {SPEED.value}")
100
101        separator(DR_SEP_2)
102
103        // ---- A combo — driven by `imgui_force_set` with the selected index ----
104        var preset_label = "(none)"
105        if (PRESET.value >= 0 && PRESET.value < length(PRESETS)) {
106            preset_label = PRESETS[PRESET.value]
107        }
108        combo_select(PRESET, (text = "Preset",
109                              preview_value = preset_label,
110                              flags = ImGuiComboFlags.None)) {
111            for (i in range(length(PRESETS))) {
112                let is_sel = (i == PRESET.value)
113                if (selectable_label(PRESET_ROW[i], PRESETS[i], is_sel)) {
114                    PRESET.value = i
115                }
116            }
117        }
118        text("PRESET = {preset_label} (idx {PRESET.value})")
119
120        separator(DR_SEP_3)
121
122        // ---- A button — fired by `imgui_click` ----
123        if (button(RUN_BTN, (text = "Run"))) {}
124        text("RUN_BTN.click_count = {RUN_BTN.click_count}")
125
126        separator(DR_SEP_4)
127
128        // ---- A popup — opened/closed via `imgui_open` / `imgui_close` ----
129        text("STATUS_POPUP - driven by imgui_open / imgui_close.")
130        popup(STATUS_POPUP, (text = "StatusPopup",
131                             flags = ImGuiWindowFlags.None)) {
132            text("Driven from outside via imgui_open.")
133            separator(DR_SEP_5)
134            text("RUN_BTN.click_count = {RUN_BTN.click_count}")
135        }
136    }
137
138    end_of_frame()
139    Render()
140    var w, h : int
141    live_get_framebuffer_size(w, h)
142    glViewport(0, 0, w, h)
143    glClearColor(0.10f, 0.10f, 0.12f, 1.0f)
144    glClear(GL_COLOR_BUFFER_BIT)
145    live_imgui_render()
146
147    live_end_frame()
148}
149
150[export]
151def shutdown() {
152    live_imgui_shutdown()
153    live_destroy_window()
154}
155
156[export]
157def main() {
158    init()
159    while (!exit_requested()) {
160        update()
161    }
162    shutdown()
163}

The command surface

Two kinds of command — faithful input (does what a user does) and bypass (does what a user can’t):

  • Raw synth IO (faithful): imgui_mouse_pos, imgui_mouse_button, imgui_mouse_play, imgui_key_press, imgui_key_type. The driver pretends to be a mouse or keyboard, feeding events into the ImGui input queue. Used by imgui_playwright for cursor-visible recordings.

  • Click a widget by name (faithful): imgui_click, imgui_focus. imgui_click resolves the widget by path (or hex_id), warps to its center, and presses/releases through ImGui’s own input path — a real click, so the widget behaves exactly as a user click would (it errors if the target isn’t rendered this frame). imgui_focus forces keyboard focus. No trajectory to script, but the widget must be on screen.

  • Write a value directly (bypass): imgui_force_set. The framework looks up the widget and queues state.has_pending = true + state.pending_value = ...; the render function submits it next frame. Does what a click can’t — an exact value, an off-screen or inactive widget.

Plus the read side and the container channel:

  • imgui_snapshot — full registry as JSON, the first call in any driver.

  • imgui_open / imgui_close — set state.pending_open / state.pending_close on container widgets (popups, closable windows, tabs, tree nodes).

Prefer imgui_click for clicks and imgui_force_set for values — the first is a faithful click, the second a deterministic value write. Drop to raw synth IO only when there’s no higher-level counterpart (drag along a custom trajectory, paste a long string into a focused input, sustain a chord, …).

imgui_snapshot — read the world

The first call every driver makes:

curl -X POST -d '{"name":"imgui_snapshot"}' localhost:9090/command

Response shape:

{
  "frame": 412,
  "globals": {
    "DRIVE_WIN": {
      "kind": "window",
      "bbox": [30, 30, 670, 490],
      "hex_id": "0x2c1a8f4b",
      "payload": { "open": true, "size": [640, 460], ... }
    },
    "DRIVE_WIN/SPEED": {
      "kind": "slider_int",
      "bbox": [...],
      "hex_id": "0x...",
      "payload": { "value": 5, "bounds": [0, 10], ... }
    },
    "DRIVE_WIN/RUN_BTN": {
      "kind": "button",
      "bbox": [...],
      "payload": { "click_count": 0 }
    },
    ...
  },
  "io": {
    "mouse_pos": [320, 180],
    "active_widget": "..."
  }
}

Use it to:

  • discover what’s on screen and what kind each widget is;

  • read bbox for L1 mouse synthesis (when needed);

  • check payload for current state (test assertions);

  • read hex_id for fallback dispatch when the path isn’t stable.

imgui_force_set — value writes

imgui_force_set is the universal value-write endpoint — slider, checkbox, combo, color, text input, dock-window position. Type-dispatched on the value’s JSON shape:

# string into a text input
curl -X POST -d '{"name":"imgui_force_set","args":{"target":"DRIVE_WIN/USER","value":"Alice"}}' \
     localhost:9090/command

# int into a slider
curl -X POST -d '{"name":"imgui_force_set","args":{"target":"DRIVE_WIN/SPEED","value":7}}' \
     localhost:9090/command

# int into a combo (selected index)
curl -X POST -d '{"name":"imgui_force_set","args":{"target":"DRIVE_WIN/PRESET","value":2}}' \
     localhost:9090/command

# array-of-floats into a color picker
curl -X POST -d '{"name":"imgui_force_set","args":{"target":"DRIVE_WIN/TINT","value":[0.2,0.7,0.4]}}' \
     localhost:9090/command

Under the hood: the registered dispatcher for the widget’s state struct unpacks the JSON, type-checks it against the state’s value field, flips has_pending = true, stores pending_value. The render function picks it up next frame; ImGui submits the new value through its own UpdateValue path.

imgui_click — fire a click

imgui_click is a real synthetic mouse click: it resolves the target to its on-screen bbox, warps the cursor to the center, and presses then releases the button across one frame — through ImGui’s own input path, so the widget can’t tell it apart from a hardware click:

curl -X POST -d '{"name":"imgui_click","args":{"target":"DRIVE_WIN/RUN_BTN"}}' \
     localhost:9090/command

The button’s render function returns true, click_count increments, and the daslang side sees both the inline if (button(...)) and RUN_BTN.clicked / RUN_BTN.click_count as expected. Pass "button": 1 for a right-click (context menus), 2 for middle. Because it’s a real click, the target must be rendered this frame — clicking an unrendered widget returns an error (use imgui_force_set to drive a widget that isn’t on screen).

imgui_open / imgui_close — containers

Containers expose an open-state channel through state.pending_open and state.pending_close. imgui_open flips pending_open; imgui_close flips pending_close. The next frame’s render function applies the change:

curl -X POST -d '{"name":"imgui_open","args":{"target":"DRIVE_WIN/STATUS_POPUP"}}' \
     localhost:9090/command
curl -X POST -d '{"name":"imgui_close","args":{"target":"DRIVE_WIN/STATUS_POPUP"}}' \
     localhost:9090/command

The same channel handles popups, closable windows, tabs (closable-tab visibility specifically — see tutorial_containers for the tab-item caveat), tree nodes, and collapsing headers.

The flow on a single command

Every command runs the same path:

  1. daslang-live HTTP server receives POST /command.

  2. Routes by name to the registered [live_command] handler.

  3. Handler looks up target in the registry’s path map (or hex_id reverse map).

  4. Either spawns a synthetic-input coroutine (imgui_click — warp + press/release over a frame) or mutates the matching state struct’s pending field (imgui_force_set / imgui_open / imgui_close); returns {"ok": true, ...} on the HTTP response.

  5. Over the next frame(s) the script’s update() runs; ImGui processes the synthetic input, or the render function applies the pending field, and the updated state is observable from the next imgui_snapshot.

So commands settle over the next frame or two by design — there’s no ambiguity about which frame’s state corresponds to a given response. For test harnesses that need to read the result, the canonical pattern is: command, then await_quiescent (waits a frame), then imgui_snapshot.

Standalone vs live

The HTTP server only exists under daslang-live. Standalone daslang.exe runs the same script but the live-command endpoints aren’t bound — drive-from-outside scenarios require the live host.

Driving from outside (recap)

A complete drive sequence for this tutorial’s app:

# Read the world
curl -X POST -d '{"name":"imgui_snapshot"}' localhost:9090/command

# Write each widget kind
curl -X POST -d '{"name":"imgui_force_set","args":{"target":"DRIVE_WIN/USER","value":"Alice"}}' \
     localhost:9090/command
curl -X POST -d '{"name":"imgui_force_set","args":{"target":"DRIVE_WIN/SPEED","value":7}}' \
     localhost:9090/command
curl -X POST -d '{"name":"imgui_force_set","args":{"target":"DRIVE_WIN/PRESET","value":2}}' \
     localhost:9090/command
curl -X POST -d '{"name":"imgui_click","args":{"target":"DRIVE_WIN/RUN_BTN"}}' \
     localhost:9090/command
curl -X POST -d '{"name":"imgui_open","args":{"target":"DRIVE_WIN/STATUS_POPUP"}}' \
     localhost:9090/command

The recording at the top of this page runs this exact sequence — just JSON commands. The value writes and container toggles flow through the state-struct pending channel with no mouse motion; the imgui_click is a real synthetic click at the button’s center.

Next steps

Now that the JSON-driven view is explicit, the visual aids tour walks through every overlay the recordings used: highlight, mouse trail, cursor sprite, narrate, key HUD, focus rect — all [live_command]-wrapped so the same curl pattern reaches them.

See also

Full source: examples/tutorial/driving_outside.das

Richer reference: examples/features/io_synth_text.das — imgui_key_type streams text as synthetic key + char events through the key timeline; the synthetic keyboard layer in action.

Snapshot contract: imgui_boost_runtime.das’s g_serializers per-kind payload definitions.

Previous tutorial: Live reload

Boost macros — the macro layer.