Distinguished Engineer, Data Platform

ABOUT THE ROLE CloudZero is growing fast. Our customer base is expanding, the data challenges we're solving are getting more complex, and the platform is scaling to match. As a Distinguished Engineer on the Data Engineering team, you'll own some of the hardest infrastructure problems at CloudZero: shaping the next-generation streaming data platform, the dimensional cost model underlying every attribution decision, the hot/cold storage architecture serving both real-time and historical queries, and the query engine that powers our entire product. This is real platform architecture work at real scale, not a consulting role or a review-and-advise job. You'll define the roadmap, drive the foundational decisions, and be a force multiplier for a talented engineering team — evolving CloudZero from batch-oriented pipelines toward a streaming-first architecture where cost attribution reaches engineers within seconds of a resource being used, not the next morning. This role is ideal for an architect who has built systems like this before, has the scars to prove it, and wants to see their decisions matter in direct and measurable ways for customers and for the business. WHAT YOU'LL DO Define the Data Platform Architecture - Lead end-to-end technical design for CloudZero's next-generation data platform, from event ingestion and stream processing through hot/cold storage and the query layer to the API surface - Document architectural decisions, tradeoffs, and migration strategies with the rigor of an RFC-driven process - Shape and drive every layer of the new architecture: event ingestion, stream processing and enrichment, real-time serving, analytical storage, query layer, and API Drive Streaming Infrastructure to Production - Design and deliver CloudZero's real-time data pipeline from ingestion through enrichment to serving - Establish SLOs for throughput, latency, and correctness, and build the operational playbooks that make this system trustworthy enough to replace the batch pipelines our entire product currently depends on - Tackle real-time streaming at scale across thousands of customers simultaneously, with fault tolerance, backpressure awareness, and correctness as non-negotiables Tackle the Dimension Cardinality Problem - Redesign CloudZero's dimensional cost model to support high-cardinality, multi-dimensional cost attribution without runaway materialization costs - Drive incremental, delta-based materialization strategies using modern open table formats, dramatically reducing expensive full-rebuild jobs and unlocking millions in annual infrastructure savings Evolve the Query Layer - Assess CloudZero's current query infrastructure, drive in-flight migrations to completion, and lead the evolution of the query engine layer going forward - Own performance optimization across partition pruning, predicate pushdown, and query planning, and set the vision for how the query layer grows as data volumes scale 10x Extend Cost Attribution to Real-Time - Evolve CloudZero's proprietary cost attribution engine from a batch-oriented model to one that assigns complex cost dimensions by team, feature, and customer within seconds of resource usage - Rethink enrichment, data lineage, and correctness guarantees in a streaming context Shape the Data Engineering Roadmap - Partner with product, infrastructure, and analytics engineering to define a multi-year data platform roadmap - Build consensus across engineering leadership on foundational investments including table formats, streaming frameworks, query engines, and schema management Elevate the Engineering Team - Participate in architecture reviews, contribute to design patterns and best practices, and mentor senior and staff engineers through code review, pairing, and structured feedback - Make everyone around you better, not by directing, but by raising the collective craft WHAT YOU BRING Data Platform & Architecture - 10+ years in data engineering with a clear trajectory toward principal or sta