Docs / SDK / Reference / Draw widgets in your own UI

Draw widgets in your own UI

What the UI-less client gives you when an agent shows a widget, which parts you can rely on, and how to draw quick replies and cards.

Use this page when you build your own interface with the UI-less client and the agent shows something other than text: a chart, a table, a card, quick replies or capability cards. It says what arrives, what you can build on, and what you can't.

The drop-in element draws all of these for you. If you use it, you don't need this page.

The short answer

Quick replies and capability cards have a small, documented shape. Draw them yourself.
Every other widget (chart, callout, table, card, badge, stat and widgets authored in AMS) arrives as a widget tree in doc. That tree is not a public contract. The SDK types it as unknown, Mindset doesn't serve a renderer for it, and its format can change without notice. Show the plain-text summary instead, or use the drop-in element.

Where widgets arrive

When	Where	Fields
During a turn	A `widget-payload` event on `chat.on()`	`doc`, `kind`, `summary`, `toolCallId`
In a restored transcript	`widget` on a message in `history_settled` or `transcript()`	`doc`, `summary`, `kind`, and sometimes `ref` and `block`

kind, summary and toolCallId are part of the public contract. So are ref ({ handle, version }, present when the widget was authored in AMS) and block (present when a script used the widget to ask a question). New fields can appear, so ignore any you don't use.

A message that showed a widget has that widget's summary as its content. Drawing role and content alone still gives an honest transcript.

Draw each kind

`kind`	What `doc` holds	What to draw
`quick_replies`	`{ replies: [{ label, value? }] }`	One button per reply, showing `label`. A tap sends `value`, or `label` when there is no `value`, as the user's next message
`capability_cards`	`{ cards: [{ title, description, prompt }] }`	One card per entry, showing `title` and `description`. A tap sends `prompt` as the user's next message
`chart`, `callout`, `table`, `card`, `badge`, `stat`, `widget`	A widget tree. Not a public contract	`summary`, as plain text

Branch on kind. Never pass doc to code that expects one shape for every kind.

A complete example

This module draws text, quick replies, capability cards and a summary for every other widget into a plain page. It uses the /api/session endpoint from Put an agent on your page.

html

<div id="log"></div>
<form id="composer">
  <input id="message" autocomplete="off" />
  <button type="submit">Send</button>
</form>

<script type="module">
  import { createAgentConversation } from "https://eu.mindset.ai/sdk/mindset-agent-uiless.js";

  const log = document.querySelector("#log");
  const form = document.querySelector("#composer");
  const input = document.querySelector("#message");

  const chat = createAgentConversation({
    agent: "billing-help",
    getSession: async () => {
      const r = await fetch("/api/session");
      if (!r.ok) throw new Error("Could not start a session");
      return r.json();
    },
  });

  function addLine(text, className) {
    const p = document.createElement("p");
    p.className = className;
    p.textContent = text;
    log.appendChild(p);
    return p;
  }

  function send(text) {
    addLine(text, "user");
    chat.send(text).catch(() => {});
  }

  function addButtons(items) {
    const row = document.createElement("div");
    for (const { label, text, detail } of items) {
      const button = document.createElement("button");
      button.type = "button";
      button.textContent = detail ? `${label}: ${detail}` : label;
      button.addEventListener("click", () => {
        row.querySelectorAll("button").forEach((b) => (b.disabled = true));
        send(text);
      });
      row.appendChild(button);
    }
    log.appendChild(row);
  }

  function drawWidget(kind, doc, summary) {
    if (kind === "quick_replies" && Array.isArray(doc?.replies)) {
      addButtons(doc.replies.map((r) => ({ label: r.label, text: (r.value ?? r.label).trim() })));
    } else if (kind === "capability_cards" && Array.isArray(doc?.cards)) {
      addButtons(doc.cards.map((c) => ({ label: c.title, detail: c.description, text: c.prompt })));
    } else if (summary) {
      addLine(summary, "widget-summary");
    }
  }

  let reply = null;

  chat.on((event) => {
    switch (event.type) {
      case "text_delta":
        reply ??= addLine("", "agent");
        reply.textContent += event.content;
        break;
      case "widget-payload":
        drawWidget(event.kind, event.doc, event.summary);
        break;
      case "complete":
      case "run_error":
        reply = null;
        break;
      default:
        break;
    }
  });

  form.addEventListener("submit", (e) => {
    e.preventDefault();
    const text = input.value.trim();
    if (!text) return;
    input.value = "";
    send(text);
  });
</script>

To draw a returning user's history the same way, read widget on each message in history_settled and call drawWidget(m.widget.kind, m.widget.doc, m.widget.summary).

What you can't do yet

Draw charts, tables, cards and authored widgets yourself, reliably. You can read the tree in doc, but nothing promises its shape. Code that parses it can break on any Mindset release.
Answer a script's question. A widget with block is a question a script is waiting on. The UI-less client has no way to answer it.
Send a widget's own actions back. widgetAction() sends an action from a rendered widget to the agent. Without a renderer you have no actions to send, except the quick reply and card taps above, which are ordinary messages.

If your interface needs these widgets, use the drop-in element, or ask the agent's author in AMS to have the agent answer in text for your integration.

Events reference for the widget-payload event.
The UI-less client for the whole client.