Sr Staff - Data Platform Engineer

WEX is a leading company in Information Technology, seeking a Senior Staff Data Platform Engineer to architect the core data infrastructure for the enterprise. This role involves building high-performance engines and frameworks that enable data engineers to operate efficiently while addressing complex data engineering challenges.

Responsibilities:

• Architectural Sovereignty: Define the 3-5 year technical roadmap for the Data Lakehouse. You aren't just using tools; you are deciding how storage, compute, and metadata layers (e.g., Apache Iceberg, Snowflake Horizon or Databricks Unity Catalog) interact at an elemental level
• Platform-as-a-Product: Build internal SDKs, CLI tools, and automated orchestration frameworks. Your goal is to abstract away cloud complexity via Control Planes and Custom Operators, allowing Data Engineers to focus on business logic rather than infrastructure boilerplate
• Internal R&D: Prototype and benchmark emerging technologies (e.g., specialized Spark extensions) to keep the platform at the bleeding edge of performance and cost-efficiency
• Global Governance & Security: Architect "compliance-by-design" systems. Automate data lineage, PII masking, and fine-grained access control across petabyte-scale environments without sacrificing developer velocity
• AI Governance Lake: Facilitate real-time data ingestion by implementing streaming support for Open Telemetry data from AI Agents into the Data Lake, drive the development of advanced AI evaluation metrics and reporting
• Engineering Excellence & Influence: Set the gold standard for code quality and system design across the company. You will lead Cross-Functional Architecture Reviews and serve as the final escalation point for the most complex system outages or performance bottlenecks
• Organizational Mentorship: Beyond individual mentoring, you will foster an "Engineering Community," influencing the hiring bar and professional development paths for the entire data engineering organization

Requirements:

• 15+ years in software engineering and distributed systems, with at least 4 years in a principal or staff-level capacity leading platform-scale initiatives
• Strong fundamentals on software engineering, system architecture, and scalable production applications (Algorithms, Data Structures, and System Design)
• Experience in the Java/J2EE ecosystem (Spring Boot, Microservices) and python
• Experience building the platforms / framework engines and APIs that power data movement, rather than just building the ETL/ELT pipelines themselves
• Deep internal knowledge of Apache Iceberg, Hudi, or Delta Lake (metadata management, manifest files, and compaction strategies)
• Experience contributing to or deeply customizing open-source data projects (e.g., Spark, dbt)
• Extensive experience designing and building high-throughput, fault-tolerant data pipelines and orchestration frameworks that ensure robust data transformations and high data quality
• Extensive experience with cloud architecture and services, including AWS (S3, EMR, Kubernetes, Lambda) and Azure
• Deep understanding of CI/CD automation, modern development tools, Git Actions, Terraform and frameworks
• Experience leveraging AI Code Gen platforms into software development lifecycle (SDLC) to automate code generation, reviews, generate unit tests, and perform root-cause analysis of system failures
• Ability to architect the infrastructure required to support AI Agent development by enabling vector database integration
• Proven track record of 'leading by influence'—driving adoption of new technologies across multiple autonomous teams
• Ability to communicate complex architectural trade-offs (e.g., 'Latency vs. Consistency' or 'Build vs. Buy') to C-suite executives and junior engineers alike
• Bachelor's or Master's degree in Computer Science (Distributed Systems focus) preferred, or equivalent deep industry experience