Streaming Structured Output Without a Flickering UI

A spinner hides uncertainty. A streaming interface puts that uncertainty on screen.

For Codelit's architecture canvas, showing the first few nodes can make the wait more useful. But once a node appears, the UI has made a promise: the user expects it to stay put, keep its identity, and explain itself if the final answer changes.

A JSON parser expects a complete value, while a network chunk can end inside a string, number, or nested object. Structured-output constraints do not make each chunk parseable. Successful schema conformance also does not establish that the content is factually or architecturally correct.

For example, a chunk ending after an opening quote is not malformed transport. It is simply unfinished JSON. Calling the ordinary parser after every chunk mostly produces expected syntax errors.

Anthropic's streaming reference describes tool input as partial JSON strings and supports accumulating them before parsing or using incremental helpers. The browser still needs a policy for what those provisional values are allowed to do.

Use a tested incremental parser or the provider SDK when it meets your requirements. A small scanner can illustrate the idea, but brace counting is not a complete JSON parser. Escapes, Unicode fragments, scalars, nesting, malformed input, and resource limits all deserve tests.

A parser can be short and still fail at exactly the chunk boundary a production stream eventually produces. Test every split point in representative documents, including escapes and nested values, before treating a hand-written closer as production infrastructure.

The useful architecture survives without a custom parser:

1. Append each chunk to the original buffer.
2. Derive a provisional object without altering that buffer.
3. Validate the provisional object's displayable fields.
4. Validate the original complete response after successful completion.

This is the same boundary I learned to respect in the Mermaid importer: parsing identifies structure; a separate validation layer decides whether that structure belongs in the product.

A partial object is safe to show when it satisfies the renderer's minimum contract and the interface clearly marks it as provisional. That threshold is intentionally weaker than final validation and must not grant permission to export, execute, or otherwise rely on incomplete data.

For a diagram node, the minimum might be a unique identifier and a supported node type. A label can use a placeholder. An edge should wait until both endpoints exist.

The generation pipeline still needs semantic checks after schema validation. A graph can have valid fields and describe an impossible dependency.

Keep component types and keys stable while content changes, and avoid applying a new layout on every fragment. Streaming introduces frequent updates; it should not repeatedly reset selection, replay entrance effects, or move the item the user is trying to read.

React ties state to component identity and position in the tree. A fresh array does not automatically remount its keyed children. A changing key can.

Use a stable item identifier, not a content hash. Array indices can work for an append-only list but become unsafe when items are inserted, removed, or reordered. Validate model-supplied IDs for uniqueness before they become render keys.

For a graph, I prefer a stable holding layout during generation and a deliberate final arrangement. If dragging is allowed during the preview, pin user-moved nodes. Otherwise make the non-editable preview state obvious.

Memoization can reduce unnecessary work, but it is not the only control. Batch updates, preserve unchanged references, and profile the React Flow canvas with representative graphs.

Reconcile the validated result by identity, separating additions, updates, and withdrawals. Decide which fields belong to the model and which belong to the user. A merge that preserves position but overwrites an edited label has still lost user work.

Keep user overrides in a separate layer or track dirty fields explicitly. Do not assume that spreading the existing object before incoming data protects every edit.

A retry may produce a different graph with entirely different IDs. In that case, presenting a clearly labeled replacement can be more honest than animating an apparently continuous update.

The difficult product decision is not how to fade out a node. It is whether a user could act on that node before the system knew it belonged there.

A completed stream has a successful protocol-level outcome and a final object that passes the required validations. End-of-file is not enough. A normal terminal event can still report truncation, refusal, or another result your application must not call success.

Anthropic also documents error events inside a stream that already began successfully. That connects directly to provider routing: an HTTP success cannot be the router's final definition of a good answer.

Withdrawn content should leave in a way that communicates correction without interrupting the rest of the interface. A short transition can help, but it is optional. Reduced-motion support, accessible status text, and predictable focus matter more than animation polish.

Do not rely only on a transition-end event to remove withdrawn nodes. Reduced motion, interrupted transitions, and unmounted components can prevent that event from being useful. State cleanup needs a reliable non-animation path.

For text-heavy or high-consequence output, waiting for a complete section may be better than showing every fragment. Streaming is a per-surface decision, not a feature switch that should apply everywhere.

Measure time to the first meaningful item, time to validated completion, interruptions, and corrections users have to absorb. A stream that paints early but constantly rearranges itself can feel slower than a stable result that arrives later.

Compare representative tasks on the same models and devices. Include long labels, slow chunks, an early disconnect, and a rejected final object. Time-to-first-token alone misses most of those costs.

If I rebuilt this feature, I would start with failure states and reconciliation, then choose the parser. The parser gets content onto the screen. The state model decides whether the user can trust what they saw.