Data Streams Product
Real-time ETL platform — managing complex product evolution
OVERVIEW
Tom Sawyer Data Streams is a schema-driven, real-time ETL platform that unifies streaming and legacy data sources, such as Apache Kafka and relational databases, into a governed, query-ready knowledge graph.
As product designer, I made this technical integration platform accessible and intuitive for data architects and engineers working at scale. I designed a clean, visually guided experience in close collaboration with developers, product managers, and data architects, in an agile environment.
THE CHALLENGE
Complexity of integration. Streaming architectures and legacy systems live in silos, and mapping them into a unified graph requires complex transformations, schema definitions, and real-time logic.
Steep technical barrier. Defining pipelines, transformations, and schema relationships typically demands deep expertise in code, ETL tools, or query languages, which slows adoption.
Lack of visibility and governance. Without a visual, schema-driven approach, enforcing data quality, lineage, and business rules in real time is difficult.
Experience friction. A purely code-based interface makes it hard to reason visually, test flows, and iterate quickly.
MY PROCESS
User Research
Built representative user personas from user profiles, and combined them with product documentation audits and competitor benchmarking to map complex integration patterns, such as Kafka and change data capture (CDC), into requirements for visual flow design and schema-driven transformations.
Problem Statement
Translated needs for data visibility and schema integrity into a requirement set: a visual flow editor, real-time previews, a SpEL transformation UI, and integrated deployment controls that ensure validated graph commits.
Test & Validate
Refined the experience through demos, internal testing, and feedback sessions with QA, product, and stakeholders, adding inline validation, schema graph views, and transformation previews to improve error messaging and complex expression editing.
Collaborate & Deliver
Took part in sprint planning, requirements reviews, design document reviews, QA, and documentation, working closely with developers, product managers, and stakeholders to design for edge cases such as malformed data.
Original screens, data, and working files from this project are confidential and can’t be shared. Process artifacts shown here are recreated to protect confidential project materials.
OUTCOME
Validated visual logic. Demos, internal testing, and feedback sessions validated the visual-first approach for complex real-time data integration.
Faster technical alignment. Design prototypes reduced architectural ambiguity and established UI patterns now adopted across the Modern Web Framework.
Feedback-driven delivery. Pre-launch feedback from the product development team and stakeholders shaped priorities and release scope, and refined the shipped product.
Strategic growth potential. Sales and engineering leadership expect the tool to become a core platform component for enterprise schema governance and ETL simplification.
SKILLS
Product design, UX/UI design, interaction design, information architecture, workflow design, prototyping, usability testing, data visualization, ETL and data integration, design systems, design tokens, enterprise SaaS, B2B data platforms, agile collaboration, cross-functional collaboration, developer handoff