Architecture leaked into every task
Users had to translate infrastructure concepts before they could make a product decision.
02 · Real-time data streaming
One experience for data in motion.
One developer experience across queuing, event streaming, connectors, and stream processing.
Public-safe reconstruction This visualization stays close to the authored DX shell, job list, and right-side agent interaction. Because the production interface is confidential, internal URLs, identifiers, topology, account data, and exact implementation details are replaced with synthetic content.
The product problem
The infrastructure was powerful. The product model was fragmented.
Application teams had to reason across queues, topics, subscriptions, connectors, processing jobs, schemas, retries, and ownership boundaries. I reframed separate tools as one journey: move data from a source to a trusted consumer, with operational context preserved end to end.
Changes had downstream effects
A configuration or schema change could affect producers, consumers, and recovery paths.
Related services felt unrelated
Queuing, Kafka, connectors, and processing used different structures for similar decisions.
Platform strategy
I designed the shared mental model before the screens.
The common product language followed the data lifecycle, not the organization chart. Teams could see what produced data, how it moved, what transformed it, who consumed it, and where responsibility changed.
Real-time data journeyGeneralized from authored platform work
01Create cluster
Environment and ownership
Environment and ownership
02Add connector
Source or destination
Source or destination
03Configure stream
Events and schema
Events and schema
04Create job
Flink or Spark processing
Flink or Spark processing
05Operate
Validate, monitor, recover
Validate, monitor, recover
The model joins queuing, Kafka-based streaming, connector services, and stream processing without exposing implementation topology.
Objects over service names
Navigation followed durable concepts such as source, stream, consumer, and job.
Dependencies before action
Ownership, schema impact, and recovery were visible before consequential changes.
One behavioral contract
Shared states and validation reduced reinvention across product teams.
Operations agent management
Agents can plan and explain. Operators remain in command of production.
I explored an operations-agent layer over the platform model: interpret infrastructure, select diagnostic tools, draft constrained configurations, identify likely failure paths, and present a reviewable recovery plan.
Translate the topology
Give the agent read-only tools to summarize producers, streams, connectors, processing jobs, and consumers.
Draft with constraints
Generate a reviewable configuration plan while exposing tool inputs, schema assumptions, and dependencies.
Orchestrate next steps
Connect errors, retry behavior, and downstream impact to a sequenced recovery plan for operator approval.
Agent boundary
An agent may inspect, explain, and recommend. It may not publish, delete, reroute, or change production resources without an accountable operator reviewing the plan, tools, scope, and impact.
Evidence, trade-offs, reflection
The workflow was shaped around real operational decisions.
Builders and operators
Application developers, data engineers, platform operators, and architects needed different levels of control while sharing one view of ownership, health, and downstream impact.
Prototype failure paths
Working prototypes and architecture reviews made cluster, connector, job, validation, diagnosis, and recovery behavior testable before teams committed to implementation.
Abstraction with an escape hatch
Hiding infrastructure made setup easier but could weaken expert control. I prioritized task language and safe defaults while preserving advanced configuration through progressive disclosure.
Instrument the operating loop
I would add explicit measures for setup completion, validation failure, time to diagnosis, recommendation acceptance, and recovery success before scaling agent autonomy.
Staff-level leverage
The deliverable was a platform operating model, not a collection of mockups.
I aligned product managers, engineers, architects, and operations around shared vocabulary and lifecycle rules. Working prototypes made behavior discussable; specifications carried decisions across teams; AI-assisted design-to-code shortened the distance from design intent to a testable implementation.
One ecosystem
Queuing, streaming, connectors, and processing became one coherent product narrative.
Stable task model
Define, connect, move, process, and operate organized the experience.
Cross-team execution
UX rules connected architecture decisions to implementation and review.
Agent strategyCross-org alignmentAgent UX architectureTool contractsOrchestration prototypingOperational guardrails
Next case