Dev Spy

Name: Dev Spy
Author: @rebind

@rebindv0.2.0

See the pointer, window, and color under it while you write a script, record observations and macros, and try AI calls.

A HUD in the corner of your screen shows what a Rebind script would see right now: the pointer position, the monitor under it, the active window, the color under the pointer, and the keys you hold. Use it while you write a script to find coordinates, colors, and window names without guessing.

Dev Spy can also record. An observation saves every key, click, scroll, and window change, with a screenshot at each action and when things settle, as a file an AI model or a script author can read. A macro recording saves only keys, mouse, and controller input, and turns it into a macro Rebind can play back. From the same menu you can try rb.ai.decide, rb.ai.find, and rb.ai.read on the screen in front of you before you put them in a script.

Needs Rebind 3.13.0 or newer on Windows or macOS. It requests the ai permission and clipboard access. On macOS, Rebind needs Screen Recording permission for screenshots and the color under the pointer. The AI console needs Rebind Pro (or its free trial); everything else is free.

Use

Press Ctrl + Alt + D (Control + Option + D on macOS) to open the menu. Press a number, or move with Up and Down and press Enter. Escape closes it. While the menu is open, those keys go to the menu only.

From the menu you can start an observation or a macro recording, stop it, mark moments while it runs, hide or show the HUD, open the AI console, copy the pointer and color as a line of Luau (such as rb.hid.move_to(1287, 644) -- #1e1e2e), and copy the full path of Dev Spy's data folder. Hiding the HUD, observing, recording a macro, and copying the pointer and color can also get their own keys under Keys in the settings; none has one by default, so Dev Spy takes no other key from your apps.

While the HUD is hidden and a recording runs, a strip at the top of the screen shows its time, its screenshots or events, and that it is logging keys. It also shows a stopped recording that is not saved yet.

Dev Spy only watches the rest of your input. Its own keys, and the keys its menu takes, are left out of recordings. The Ctrl and Alt of the menu chord stay in an observation's log, marked as Dev Spy's, and are left out of saved macros.

What the HUD shows

pointer: screen coordinates, then the same point relative to the active window.
monitor: which monitor the pointer is on, its size, and where it sits on the desktop.
window and process: the active window's title, its app, its size, and its position.
pixel: the color under the pointer as hex and as red, green, and blue. Over Dev Spy's own HUD or AI panel it reads "under Dev Spy" instead.
keys, mods, and last: the keys held now, the modifiers held, and the last input with how long ago it was.
device and pad: in hardware mode, the device the last input came from and any controller stick or trigger that is moved. Software mode cannot tell devices apart.
timer: how often Dev Spy's own timer actually runs, to the nearest 10 ms. It reads 100 ms when Rebind keeps up.

The top row shows how long Dev Spy has been running. During a recording, the HUD shows the recording's time, its events and marks, its screenshots, crops, and their size, and the size of its log.

Recordings

An observation and a macro recording log every key you press in every app, including passwords and messages, for as long as they run. The HUD and the strip say so while one is running. Stop recording before you type anything private, and check a recording before you share it.

A recording goes into the sessions folder in Dev Spy's data folder, outside your scripts folder, written as it happens, so Dev Spy keeps very little in memory. Each one is a folder with events.jsonl, one event per line. An observation adds a frames folder of PNG screenshots and a crops folder with a sharp image around each click. Each key event names the latest screenshot taken before it.

An observation takes a screenshot:

when it starts;
at once when you click, press Enter, Escape, or Tab, or use a shortcut with Ctrl, Cmd, or Alt;
at the first key press, scroll, mouse movement, or stick motion after a pause, showing what you reacted to;
when things settle: 400 ms after you stop moving the mouse, scrolling, typing, or moving a controller stick, and after each action above, so the result is on screen;
250 ms after the active window changes, including its title, position, or size, and at most once a second while it keeps changing;
after a quiet stretch, 1 second by default, so a change nobody caused is placed within about a second.

A screenshot identical to the one before it is not saved again; the log points to the earlier file, so a still screen costs almost nothing. A screenshot taken for a click or key names that input. Each click also saves a 160 by 160 crop around the pointer at the display's full resolution, which a script can search for with rb.screen.find_image, and the log notes the color at each click before and after, for up to 8 clicks in one burst. A click made while the previous crop is still being taken is logged without a crop.

What this recording is for in the settings is saved at the start of each recording. While a recording runs, the menu opens with Mark this moment, Mark success, Mark failure, and Mark with clipboard text, so an AI can tell the steps and the outcomes apart. A mark is timed when you opened the menu.

When you stop, open the menu to save it:

Save to observations/ moves it to observations/<name> in the data folder.
Save to macros/ moves it to macros/<name> in the data folder and also writes macros/<name>.luau. Key presses, mouse buttons, pointer movement, and scrolling play back, and pointer movement is replayed as the steps you made from where the recording started. A key still down when the recording stopped is let go at the end. Controller buttons play back on a Rebind Link whose USB profile has a joypad. Stick and trigger movement stays in the log only.
Name from clipboard takes the name from the text you copied. Otherwise the name comes from Name for the next save in the settings, or the time the recording started. A name already in use gets a number added, and the menu shows the name it will use.
Discard deletes the recording after you confirm.

The HUD and the save notice show paths starting with @data/, which is how Dev Spy's code names its data folder. To run a macro on its own or load it from another script, copy its .luau into your scripts folder.

A recording you have not saved is kept. If the script restarts, including after you change one of its keys, Dev Spy says so and the menu offers it again.

AI console

Open AI console from the menu. Each call runs once, and the result shows in the AI panel at the bottom left with how long it took.

Decide asks the question from the settings and picks one of the choices, with the model's confidence. It looks at the display under the pointer, or at the window you name.
Find looks for what you describe and marks the spot it found for 5 seconds.
Read at pointer reads the text in a 400 by 160 box around the pointer. Read in window reads in the window you name, or the active window.
Question from clipboard uses the text you copied as the question or description instead of the settings.

During a recording, each call and its answer are saved in the log. Each call counts against your monthly AI allowance.

Settings

Keys: the menu key, and optional keys for hiding the HUD, observing, recording a macro, and copying the pointer and color. A change restarts the script, which stops a recording that is running.
HUD: the theme, and how often the HUD refreshes.
Sessions: how long a quiet stretch lasts before an observation takes a screenshot anyway (0 turns this off), the screenshot size, when a recording stops on its own (30 minutes by default), what a macro recording includes, what the recording is for, and the name for the next save.
AI: the question and choices for Decide, what Find looks for, what Read reads, and the window to look at.

Limits

A recording stops at 512 MB, crops included, or at the time limit, whichever comes first. One screenshot is taken at a time. Settle, window, and idle screenshots stay at least a quarter second apart, while a click, an action key, or the start of a burst is taken as soon as the one before it finishes.
A window whose title keeps changing, such as a terminal with a spinner, adds about one screenshot a second while it changes.
A macro is written from a log up to 4 MB. A longer recording still saves its log, without the .luau.
A macro moves the pointer by the steps you made, so clicks land where you made them only when it starts where the recording started, on the same monitor layout. In software mode on macOS the steps are exact; through a Rebind Link or on Windows, pointer acceleration can shift them.
A game in exclusive full-screen mode on Windows hides the HUD and the menu.
Recordings stay in Dev Spy's data folder when Dev Spy updates from the marketplace. Deleting Dev Spy in Rebind deletes them, and cloud backups leave them out, so copy the ones you want to keep.
Live use is checked on macOS. Windows remains unverified.

Stop

Stop the package from the editor toolbar, or press Left Ctrl + Left Alt + K (Left Control + Left Option + K on macOS) to stop all scripts.

Version 0.2.0 (current)

hud.luau

-- What the developer sees: the HUD, the status strip that replaces it while
-- it is hidden, the AI panel, the marker on a found point, and toasts. Each
-- surface is redrawn only when its text changes.

local probe = require("probe")
local session = require("session")

local M = {}

local W, PAD, SIZE, ROW, LABEL = 380, 14, 13, 20, 90

local cfg
local hud, hud_key, hud_h = nil, nil, 0
local strip, strip_key, strip_theme = nil, nil, nil
local panel, panel_view, panel_key, panel_h = nil, nil, nil, 0
local marker = nil
local started = 0

-- the HUD is off and the strip stands in for it during a recording
M.hidden = false

-- a script has at most 16 overlay surfaces and one more raises; a surface
-- that cannot be made is logged and skipped, and not tried again for 5 s
local retry_at = {}
local function make(name, fn, ...)
  if (retry_at[name] or 0) > rb.system.time() then
    return nil
  end
  local ok, handle = pcall(fn, ...)
  if ok then
    return handle
  end
  retry_at[name] = rb.system.time() + 5000
  rb.log.warn("Dev Spy: overlay not shown: " .. tostring(handle))
  return nil
end

function M.init(config)
  cfg = config
  started = rb.system.time()
end

local function theme()
  return rb.overlay.themes[cfg.theme] or rb.overlay.themes.rebind
end

-- formatting -----------------------------------------------------------------

function M.clock(ms)
  local s = math.max(0, math.floor(ms / 1000))
  return string.format("%02d:%02d:%02d", s // 3600, s // 60 % 60, s % 60)
end

-- the whole seconds the running and recording clocks show at now, so each
-- clock shows every second at any refresh setting
function M.seconds(now)
  local s = session.live
  return math.floor((now - started) / 1000),
    s and math.floor((now - s.started) / 1000)
end

function M.size(bytes)
  if bytes < 1024 * 1024 then
    return string.format("%d KB", math.ceil(bytes / 1024))
  end
  return string.format("%.1f MB", bytes / (1024 * 1024))
end

function M.commas(n)
  local text = tostring(math.floor(n))
  local out =
    string.reverse((string.gsub(string.reverse(text), "(%d%d%d)", "%1,")))
  return (string.gsub(out, "^,", ""))
end

-- whole seconds, so the panel is not redrawn every refresh after an input
local function ago(ms)
  if ms < 1000 then
    return "<1 s ago"
  elseif ms < 60000 then
    return string.format("%d s ago", math.floor(ms / 1000))
  end
  return string.format("%d min ago", math.floor(ms / 60000))
end

-- one line in the given width, cut by characters so no UTF-8 character is
-- split; control characters such as newlines become spaces
local function fit(text, width, bold)
  width = width or W - 2 * PAD - LABEL
  text = string.gsub(tostring(text), "%c", " ")
  if rb.overlay.measure(text, SIZE, bold) <= width then
    return text
  end
  local n =
    math.max(1, math.floor(width / rb.overlay.measure("M", SIZE, bold)) - 1)
  local cut = utf8.offset(text, n + 1)
  return string.sub(text, 1, (cut or #text + 1) - 1) .. "…"
end

M.fit = fit

-- lines of at most width, broken at spaces where possible, at most max
local function wrap(text, width, max)
  text = string.gsub(tostring(text), "%c", " ")
  local per = math.max(8, math.floor(width / rb.overlay.measure("M", SIZE)))
  local lines = {}
  while #lines < max and text ~= "" do
    if (utf8.len(text) or #text) <= per then
      table.insert(lines, text)
      text = ""
    else
      local cut =
        string.sub(text, 1, (utf8.offset(text, per + 1) or #text + 1) - 1)
      local head = string.match(cut, "^(.*%S)%s+%S*$") or cut
      table.insert(lines, head)
      text = string.match(string.sub(text, #head + 1), "^%s*(.*)$")
    end
  end
  if text ~= "" then
    lines[max] = fit(lines[max] .. " " .. text, width)
  end
  return lines
end

-- drawing --------------------------------------------------------------------

local function span(text, role, bold)
  return { text = text, color = role or "text", bold = bold == true }
end

-- spans name theme roles; surfaces take colors
local function paint(t, spans)
  local out = {}
  for i, s in spans do
    out[i] = { text = s.text, color = t[s.color] or s.color, bold = s.bold }
  end
  return out
end

local function signature(lines)
  local parts = { cfg.theme }
  for _, l in lines do
    table.insert(parts, l.label or "")
    table.insert(parts, l.swatch or "")
    for _, s in l.spans do
      table.insert(parts, s.text .. s.color)
    end
  end
  return table.concat(parts, "\1")
end

local function frame(t, w, h)
  return {
    kind = "rect",
    w = w,
    h = h,
    fill = t.surface,
    border = t.border,
    border_width = t.border_width,
    radius = t.radius,
    shadow = t.shadow,
  }
end

local function divider(t, ops, y, w)
  table.insert(ops, {
    kind = "rect",
    x = PAD,
    y = y + 6,
    w = w - 2 * PAD,
    h = 1,
    fill = t.border,
  })
  return y + 14
end

local function text_op(t, x, y, spans)
  return {
    kind = "text",
    x = x,
    y = y + 1,
    size = SIZE,
    spans = paint(t, spans),
  }
end

-- the HUD --------------------------------------------------------------------

local function pointer_rows(r)
  local w, value = r.window, W - 2 * PAD - LABEL
  local rows = {
    {
      label = "pointer",
      -- a fixed width, so "win" does not move as the numbers grow
      spans = {
        span(string.format("%-12s", r.x .. ", " .. r.y)),
        span(string.format("win %d, %d", r.x - w.x, r.y - w.y), "dim"),
      },
    },
  }
  local m = r.monitor
  table.insert(rows, {
    label = "monitor",
    spans = m
        and {
          span(m.index .. " of " .. r.count),
          span(
            string.format("  %d×%d at %d, %d", m.width, m.height, m.x, m.y),
            "dim"
          ),
        }
      or { span(r.count == 0 and "no displays found" or "off screen", "dim") },
  })
  table.insert(rows, {
    label = "window",
    spans = {
      w.title ~= "" and span(fit(w.title, value)) or span("none", "dim"),
    },
  })
  local where = string.format("  %d×%d at %d, %d", w.width, w.height, w.x, w.y)
  table.insert(rows, {
    label = "process",
    spans = {
      span(fit(w.process, value - rb.overlay.measure(where, SIZE))),
      span(where, "dim"),
    },
  })
  if r.covered then
    table.insert(
      rows,
      { label = "pixel", spans = { span("under Dev Spy", "dim") } }
    )
  elseif r.hex ~= nil then
    local rgb = string.format(
      "  %d %d %d",
      tonumber(string.sub(r.hex, 1, 2), 16),
      tonumber(string.sub(r.hex, 3, 4), 16),
      tonumber(string.sub(r.hex, 5, 6), 16)
    )
    table.insert(rows, {
      label = "pixel",
      swatch = "#" .. r.hex,
      spans = {
        span("#" .. r.hex, "text", true),
        span(rgb, "dim"),
      },
    })
  else
    table.insert(
      rows,
      { label = "pixel", spans = { span("not readable", "dim") } }
    )
  end
  local keys = table.concat(r.keys, "  ")
  table.insert(rows, {
    label = "keys",
    spans = { keys ~= "" and span(fit(keys, value)) or span("none", "dim") },
  })
  local mods = {}
  for _, name in { "ctrl", "shift", "alt", "win" } do
    if r.mods[name] then
      table.insert(mods, name)
    end
  end
  table.insert(rows, {
    label = "mods",
    spans = {
      #mods > 0 and span(table.concat(mods, " ")) or span("none", "dim"),
    },
  })
  local last = r.last
  table.insert(rows, {
    label = "last",
    spans = last and {
      span(fit(last.text, value - 110)),
      span("  " .. ago(r.now - last.at), "dim"),
    } or { span("none", "dim") },
  })
  local device = last and last.device
  local spans
  if not r.hardware then
    spans = { span("none (software mode)", "dim") }
  elseif device ~= nil then
    local name = device.nickname or device.name
    spans = { span(fit(name, value - 90)), span("  " .. device.kind, "dim") }
  else
    spans = { span("no input yet", "dim") }
  end
  table.insert(rows, { label = "device", spans = spans })
  if r.axes ~= nil then
    local moved = {}
    for _, name in probe.AXES do
      local v = r.axes[name]
      if math.abs(v) >= 0.05 then
        table.insert(moved, string.format("%s %.2f", name, v))
      end
    end
    table.insert(rows, {
      label = "pad",
      spans = {
        #moved > 0 and span(table.concat(moved, "  "))
          or span("at rest", "dim"),
      },
    })
  end
  -- the script's measured timer interval: about 100 ms when the engine keeps
  -- up. Rounded to 10 ms, so its jitter does not redraw the HUD every tick.
  table.insert(rows, {
    label = "timer",
    spans = {
      span(string.format("%d ms", math.floor(r.interval / 10 + 0.5) * 10)),
    },
  })
  return rows
end

local function what_macro()
  local parts = { "keys" }
  if cfg.macro_moves or cfg.macro_scroll then
    table.insert(parts, "mouse")
  end
  if cfg.macro_pad then
    table.insert(parts, "pad")
  end
  return table.concat(parts, ", ")
end

local function session_rows(now)
  local s = session.live
  if s ~= nil then
    local observe = s.mode == "observe"
    local log = s.log_bytes + s.buf_bytes
    local rows = {
      {
        spans = {
          span(
            observe and "● OBSERVING" or "● RECORDING MACRO",
            "danger",
            true
          ),
          span("  " .. M.clock(now - s.started)),
        },
      },
      -- every key typed in any app goes into the log, passwords included
      { spans = { span("records every key you type", "warning", true) } },
      {
        spans = {
          span("events ", "dim"),
          span(M.commas(s.events)),
          span("  marks ", "dim"),
          span(M.commas(s.marks or 0)),
          observe and span("") or span("  " .. what_macro(), "dim"),
        },
      },
    }
    if observe then
      table.insert(rows, {
        spans = {
          span("screenshots ", "dim"),
          span(M.commas(s.frames)),
          span("  crops ", "dim"),
          span(M.commas(s.crops)),
          span("  " .. M.size(s.frame_bytes), "dim"),
        },
      })
    end
    table.insert(rows, { spans = { span("log ", "dim"), span(M.size(log)) } })
    table.insert(rows, { spans = { span(fit(s.dir, W - 2 * PAD), "faint") } })
    return rows
  end
  local p = session.unsaved[#session.unsaved]
  if p ~= nil then
    local kind = p.mode == "macro" and "macro" or "observation"
    local more = #session.unsaved > 1
        and string.format("  +%d more", #session.unsaved - 1)
      or ""
    return {
      {
        spans = {
          span("■ STOPPED", "warning", true),
          span(
            "  "
              .. kind
              .. (p.ended and " " .. M.clock(p.ended - p.started) or "")
          ),
        },
      },
      {
        spans = {
          -- a macro takes no screenshots: its events say how big it is
          span(p.mode == "macro" and "events " or "screenshots ", "dim"),
          span(
            M.commas(p.mode == "macro" and (p.events or 0) or (p.frames or 0))
          ),
          span(
            "  " .. M.size((p.log_bytes or 0) + (p.frame_bytes or 0)),
            "dim"
          ),
        },
      },
      {
        spans = {
          span("not saved: " .. cfg.menu_key .. " to save", "warning", true),
        },
      },
      {
        spans = {
          span(
            fit(p.dir, W - 2 * PAD - rb.overlay.measure(more, SIZE)) .. more,
            "faint"
          ),
        },
      },
    }
  end
  return {
    { spans = { span("no recording", "dim") } },
    { spans = { span(cfg.menu_key .. " opens the menu", "dim") } },
  }
end

local function draw_hud(r)
  local t = theme()
  local up = "running " .. M.clock(r.now - started)
  local rows, block = pointer_rows(r), session_rows(r.now)
  local all = { { spans = { span(up) } } }
  for _, l in rows do
    table.insert(all, l)
  end
  for _, l in block do
    table.insert(all, l)
  end
  local key = signature(all)
  if hud ~= nil and key == hud_key then
    return
  end
  hud_key = key

  local ops, y = { false }, PAD
  table.insert(
    ops,
    -- no dot here: a dot only ever means recording
    text_op(t, PAD, y, { span("Dev Spy", "text", true) })
  )
  table.insert(
    ops,
    text_op(t, W - PAD - rb.overlay.measure(up, SIZE), y, { span(up, "dim") })
  )
  y = divider(t, ops, y + ROW, W)
  for _, l in rows do
    table.insert(ops, text_op(t, PAD, y, { span(l.label, "dim") }))
    local x = PAD + LABEL
    if l.swatch ~= nil then
      table.insert(ops, {
        kind = "rect",
        x = x,
        y = y + 3.5,
        w = 12,
        h = 12,
        fill = l.swatch,
        border = t.border,
        border_width = 1,
      })
      x += 20
    end
    table.insert(ops, text_op(t, x, y, l.spans))
    y += ROW
  end
  y = divider(t, ops, y, W)
  for _, l in block do
    table.insert(ops, text_op(t, PAD, y, l.spans))
    y += ROW
  end
  local height = y + PAD
  hud_h = height
  ops[1] = frame(t, W, height)
  if hud == nil then
    hud = make("hud", rb.overlay.surface, {
      anchor = "top-left",
      display = "active",
      offset_x = 16,
      offset_y = 16,
      width = W,
      height = height,
      ops = ops,
    })
  elseif not pcall(hud.set, hud, { height = height, ops = ops }) then
    -- removed first, so a failed update never leaves a second HUD behind;
    -- a new one is made on the next refresh
    pcall(hud.remove, hud)
    hud, hud_key = nil, nil
  end
end

-- the strip ----------------------------------------------------------------

local function strip_spans(now)
  local s = session.live
  if s ~= nil then
    local tail = s.mode == "observe"
        and string.format(
          "   %s screenshots  %s",
          M.commas(s.frames),
          M.size(s.frame_bytes)
        )
      or string.format("   %s events", M.commas(s.events))
    return {
      {
        s.mode == "observe" and "● OBSERVING " or "● RECORDING MACRO ",
        "danger",
        true,
      },
      { M.clock(now - s.started) },
      { tail, "dim" },
      { "   keys logged", "warning", true },
    }
  end
  local p = session.unsaved[#session.unsaved]
  if p ~= nil then
    return {
      { "■ STOPPED ", "warning", true },
      { p.ended and M.clock(p.ended - p.started) or "--:--:--" },
      { "   " .. cfg.menu_key .. " to save", "warning", true },
    }
  end
  return nil
end

local function draw_strip(now)
  local spans = M.hidden and strip_spans(now) or nil
  if spans == nil or strip_theme ~= cfg.theme then
    if strip ~= nil then
      strip.remove()
      strip, strip_key = nil, nil
    end
    if spans == nil then
      return
    end
  end
  local key = cfg.theme
  for _, s in spans do
    key ..= "\1" .. s[1]
  end
  if key == strip_key then
    return
  end
  strip_key, strip_theme = key, cfg.theme
  if strip == nil then
    strip = make("strip", rb.overlay.status, spans, { theme = cfg.theme })
    if strip == nil then
      strip_key = nil
    end
  elseif not pcall(strip.set, spans) then
    pcall(strip.remove)
    strip, strip_key = nil, nil
  end
end

function M.draw(r)
  if M.hidden then
    if hud ~= nil then
      pcall(hud.remove, hud)
      hud, hud_key = nil, nil
    end
  else
    draw_hud(r)
  end
  draw_strip(r.now)
end

-- toasts, the AI panel and the marker ----------------------------------------

function M.toast(text, meta)
  make(
    "toast",
    rb.overlay.toast,
    text,
    { title = "Dev Spy", meta = meta, theme = cfg.theme }
  )
end

-- the AI panel uses the HUD's width and label column
local function panel_rows(v, now)
  local head = {
    span(
      "rb.ai." .. v.call .. (v.error and " failed" or ""),
      v.error and "danger" or "text",
      true
    ),
  }
  if v.ms == nil then
    table.insert(
      head,
      span(string.format("  waiting %.1f s", (now - v.started) / 1000), "dim")
    )
  else
    table.insert(head, span(string.format("  %.1f s", v.ms / 1000), "dim"))
  end
  local rows = { { spans = head } }
  if v.error ~= nil then
    for _, l in wrap(v.error, W - 2 * PAD, 4) do
      table.insert(rows, { spans = { span(l, "danger") } })
    end
  else
    table.insert(
      rows,
      { label = "question", spans = { span(fit(v.question)) } }
    )
    if v.ms == nil then
      table.insert(
        rows,
        { label = "where", spans = { span(fit(v.where), "dim") } }
      )
    else
      for i, l in v.answer do
        table.insert(rows, {
          label = i == 1 and "answer" or "",
          spans = {
            span(fit(l.text), l.role or "positive", l.role == nil),
            l.tail and span(l.tail, "dim") or nil,
          },
        })
      end
      if v.note ~= nil then
        table.insert(
          rows,
          { label = v.note[1], spans = { span(fit(v.note[2]), "dim") } }
        )
      end
    end
  end
  table.insert(rows, {
    spans = {
      -- whether this call went into a recording, fixed when it answered
      span(
        (v.logged or (v.logged == nil and session.live ~= nil))
            and "logged to the recording"
          or "not logged: no recording",
        "dim"
      ),
    },
  })
  return rows
end

local function draw_panel(now)
  local v = panel_view
  if v == nil then
    return
  end
  local t = theme()
  local rows = panel_rows(v, now)
  local key = signature(rows)
  if panel ~= nil and key == panel_key then
    return
  end
  panel_key = key
  local ops, y = { false }, PAD
  for _, l in rows do
    if l.label ~= nil then
      table.insert(ops, text_op(t, PAD, y, { span(l.label, "dim") }))
      table.insert(ops, text_op(t, PAD + LABEL, y, l.spans))
    else
      table.insert(ops, text_op(t, PAD, y, l.spans))
    end
    y += ROW
  end
  local height = y + PAD
  panel_h = height
  ops[1] = frame(t, W, height)
  if panel == nil then
    panel = make("panel", rb.overlay.surface, {
      anchor = "bottom-left",
      display = "active",
      offset_x = 16,
      offset_y = 16,
      width = W,
      height = height,
      ops = ops,
    })
  elseif not pcall(panel.set, panel, { height = height, ops = ops }) then
    pcall(panel.remove, panel)
    panel, panel_key = nil, nil
  end
end

-- view: call, question, where, started; then ms and answer lines, a note,
-- or an error
function M.ai(view)
  panel_view = view
  draw_panel(rb.system.time())
end

-- a waiting call's clock, and a theme change
function M.ai_tick(now)
  draw_panel(now)
end

-- whether a HUD or strip is still on screen, so the timer keeps drawing
-- until a hidden HUD or a finished strip is taken down
function M.showing()
  return hud ~= nil or strip ~= nil
end

function M.ai_close()
  panel_view = nil
  if panel ~= nil then
    pcall(panel.remove, panel)
    panel, panel_key = nil, nil
  end
end

-- whether a point on monitor m lies under the HUD or the AI panel, judged
-- generously from their anchors: the menu bar and the Dock sit between the
-- display edge and the work area the anchors use
function M.covers(x, y, m)
  if m == nil then
    return false
  end
  local left = x < m.x + 16 + W + 8
  if hud ~= nil and left and y < m.y + 16 + hud_h + 48 then
    return true
  end
  return panel ~= nil and left and y > m.y + m.height - 16 - panel_h - 110
end

-- a ring on a point rb.ai.find returned, for 5 seconds
function M.marker(x, y)
  if marker ~= nil then
    marker:remove()
  end
  local t = theme()
  local ring = make("marker", rb.overlay.surface, {
    x = x - 14,
    y = y - 14,
    width = 28,
    height = 28,
    ops = {
      {
        kind = "rect",
        w = 28,
        h = 28,
        border = t.accent,
        border_width = 2,
        radius = 14,
      },
    },
  })
  marker = ring
  rb.after(5000, function()
    if ring ~= nil and marker == ring then
      ring:remove()
      marker = nil
    end
  end)
end

return M