Staff Data Platform Engineer

Role Overview

• Serve as the technical authority for data platform and tracking architecture—guiding design decisions, tool selection, and engineering best practices.
• Design a next-generation data & AI platform that supports near-real-time ingestion, enrichment, and model-driven product features.
• Translate AI Engineering requirements into clear specifications and capability roadmaps through spec-driven development.
• Define and implement modern data governance: data quality, lineage, ownership, observability, and cost visibility.
• Evaluate, consolidate, or replace redundant technologies, introducing scalable SaaS solutions where appropriate.
• Own and extend core AI-platform components, including:
• Arize AI (evals & observability)
• LLM Gateway (model routing, policies, cost controls)
• Pinecone (vector database)
• Model access APIs
• OTEL collector for trace processing.
• Take full ownership of the AI Infrastructure components currently run by AI Engineering—ensuring reliability, scale, and clear fallback strategies.
• Document existing workflows across Analytics Engineering, Data Engineering, and AI Engineering to identify automation and process improvements.
• Introduce streamlined development pathways for AI—templates for CI/CD, GitOps, Python microservices, and AI-specific libraries.
• Implement FinOps principles: baseline costs, identify inefficiencies, and reduce spend across data and AI workloads.
• Support the transition toward a more self-serve model through ADRs, RFCs, and well-defined ownership guidelines.
• Mentor Data Engineers on modern Data Platform concepts—streaming, governance, observability, platform reliability, and AI-aligned workflows.
• Partner with Platform Engineering, TechOps, Data Science, and AI Engineering to drive alignment and cohesive infrastructure strategy.
• Define playbooks and guidelines that standardize how teams build pipelines, collaborate with Platform, and deliver AI-ready data.
• Increase trust across the organization through clear documentation, architecture records, and consistent communication.

Requirements

• Several years of experience as a software engineer or platform engineer working with large-scale data systems, streaming architectures, or data/AI platforms.
• Proven experience designing and delivering data platforms, including real-time ingestion, enrichment, data warehousing, and lake/lakehouse architectures.
• Expertise with streaming technologies (Kafka or similar), reverse-ETL, and modern big-data tooling.
• Hands-on experience with platform components such as Terraform, Terragrunt, CI/CD pipelines, Python, SQL, and cloud-native infrastructure.
• Experience partnering with Data Science, AI Engineering, and cross-functional teams to translate requirements into scalable platform capabilities.
• Strong communicator with the ability to influence stakeholders and document architecture clearly.
• Experience mentoring, coaching, or leading engineers technically (even without direct reporting lines)

Tech Stack

• Cloud
• ETL
• Kafka
• Microservices
• Python
• SQL
• Terraform

Benefits

• Flexible remote work arrangements
• Professional development opportunities