Lead Data Engineer

Empower is a financial services company focused on helping customers achieve financial freedom. The Lead Data Engineer will provide technical leadership for scalable data pipelines, real-time ingestion frameworks, streaming solutions, and enterprise data platforms while ensuring performance, reliability, security, quality, and governance. The role also mentors engineers, guides technical decisions, and collaborates with architects and stakeholders on complex data initiatives.

Responsibilities:

• Lead the design and implementation of advanced data pipelines, ingestion frameworks, and real-time streaming solutions
• Build, tune, and maintain scalable Kafka producers and consumers using Python, Java, Go, or Scala to support sub-second latency and high availability across distributed services
• Configure and manage enterprise change data capture (CDC) solutions, including Debezium, AWS DMS, or Qlik Replicate, across relational and NoSQL source systems such as PostgreSQL, MySQL, MongoDB, and Oracle into Kafka topics
• Develop stream-processing logic using Apache Flink, Spark Streaming, Kafka Streams, or dbt-mesh architectures to enrich, filter, and aggregate event payloads in flight
• Implement and enforce schema evolution policies using Confluent Schema Registry with Avro, Protobuf, or JSON Schema, and support appropriate delivery guarantees and idempotency across consumers
• Integrate automated code generation, validation agents, and LLM-driven orchestration using technologies such as LangGraph, AutoGen, or CrewAI to support schema drift, unstructured event parsing, and dynamic routing within event streams
• Instrument pipelines with end-to-end monitoring and alerting using tools such as Prometheus, Grafana, or Datadog, and manage consumer group offsets, partition rebalancing, and dead-letter queues
• Guide the technical execution of complex data initiatives across teams and translate highly complex business requirements into scalable data solutions
• Ensure data platforms are optimized for performance, scalability, reliability, and high availability
• Enforce data governance, security, quality, data contract, and schema management standards
• Mentor engineers, provide technical leadership on projects, and lead code reviews to ensure adherence to engineering standards
• Collaborate with architects and stakeholders to align solutions with enterprise direction and contribute to architectural and design decisions
• Drive improvements in automation, efficiency, and platform capabilities
• Complete other duties as assigned

Requirements:

• Applicants must be authorized to work for any employer in the U.S. We are unable to sponsor or take over sponsorship of an employment visa at this time, including CPT/OPT.
• • Bachelor's degree in computer science or a related field, or an equivalent combination of training and experience
• • 6-8+ years of data engineering experience with enterprise scale platforms
• • 6-8+ years of hands-on experience with Apache Kafka, Confluent Cloud, AWS Kinesis, or Apache Pulsar
• • Advanced proficiency in Python, PySpark, and SQL
• • Proficiency in Python, Java, Scala, or Go, combined with production experience using Flink, Kafka Streams, or Spark Structured Streaming
• • Strong experience with AWS and modern data platforms
• • Deep expertise with CDC tools such as Debezium and Kafka Connect and log-based replication mechanisms such as Oracle Redo, PostgreSQL WAL, and MySQL binlog
• • Hands-on experience sinking streaming events into modern lakehouse formats such as Apache Iceberg and Delta Lake and cloud warehouses such as Snowflake or BigQuery
• • Solid foundation in Docker, Kubernetes, Terraform, and CI/CD automation for event-driven microservices
• • Experience implementing data contracts and schema management using tools such as Schema Registry to support end-to-end data quality, schema evolution, and interface reliability
• • Deep understanding of data modeling, ETL/ELT processes, and data warehousing
• • Experience leading technical initiatives and mentoring engineers
• • Strong problem-solving and communication skills
• • Experience working in Agile environments
• This job operates in a professional office environment
• For remote and hybrid positions you will be required to provide reliable high-speed internet with a wired connection as well as a place in your home to work with limited disruption. You must have reliable connectivity from an internet service provider that is fiber, cable or DSL internet. Other necessary computer equipment, will be provided. You may be required to work in the office if you do not have an adequate home work environment and the required internet connection.
• • Experience integrating AI agents to self-heal pipeline failures, dynamically resolve schema drift, and extract structured data from unstructured event streams
• • Experience designing resilient fallback strategies, including automated dead-letter reprocessing, circuit breakers, and deterministic idempotency keys
• • Experience championing domain-driven event design, including Event Sourcing and CQRS, and defining enterprise-wide event contracts and governance
• • Experience establishing and formalizing data contracts between producers and consumers and leading adoption of schema registry practices for lifecycle versioning and backward and forward compatibility