Agent UI Needs Its D3 Moment
Two things hit my reading list this week from opposite directions and landed on the same gap. OpenAI shipped "Intelligent UI" to every ChatGPT user, free tier included: GPT-6 responses now render interactive elements inside the chat surface, alongside the markdown. Meanwhile, a dev.to roundup of AI chat UI libraries graded the whole field and found streaming support everywhere and agent controls almost nowhere. One product announcement, one unglamorous audit, and they agree: the chat surface is becoming an app surface, and the components you'd need to build apps inside it are not on any shelf.
Start on the left of that diagram: the events an agent runtime actually produces. Token deltas, tool calls, approval requests, traces, cost. The middle layer: state. This part is built. LangChain's frontend SDK exposes tool-call lifecycle, interrupts, checkpoint history, time-travel, and state forking through useStream(), and the Vercel AI SDK covers the same ground. The right side of the diagram is the gap: approval flows that wait on a human, trace viewers that expand a reasoning step, tool inspectors that show a call mid-flight, cost meters that tick while the run is still burning tokens. The dev.to roundup's category list is the closest thing to a market survey here, and almost no library ships them. Every product team hand-rolls its own approval widget, and throws it away at the next redesign, because the component mutates with every UX experiment.
The analogy that keeps occurring to me is dataviz in 2011. D3 shipped scales, selections, axes: correct, composable, unusable without ceremony. Then NVD3, Plotly, and Vega-Lite wrapped them, and nobody has hand-rolled an axis in a decade. Agent UI sits at the pre-wrapper stage of that story, with one difference the diagram can't show: a mis-drawn axis is ugly, but a mis-drawn approval flow leaks data or destroys state. The correctness stakes are higher at this layer, which is exactly why it should be a library's job and not each product team's weekend.
So my next side-quest is a small library of these primitives: an <ApprovalFlow> parameterized by what "yes" means per domain, a trace viewer that treats tokens-for-time as a rendering budget, a tool-call inspector that reads useStream() state rather than keeping its own replica. One question I haven't answered: should primitives bind to provider state shapes like useStream or the Vercel AI SDK, or define their own protocol with adapters on top? Binding is fast to build and couples the library to whatever ships next month. Owning the protocol is slower up front and survives provider churn. I lean toward owning it, but I want a week building against useStream before I commit to that in public.
Streaming is solved. Every library in that roundup has it, which is why the roundup grades on it and moves on. The unsolved layer is controls: who approves what, who sees what, and who can steer a run after it starts. That is where the hand-rolling happens, and it is the part worth turning into a library.