Key skills
AirflowDistributed SystemsETLKafkaPythonSparkSQLAIMLMLOpsData EngineeringData LakeAnalyticsSnowflakeDatabricksRemote Work
About this role
Role Overview
- Architect and write software that solves complex business problems.
- Own and serve as a Subject Matter Expert (SME) for large software systems.
- Continuously monitor data pipelines in production and implement automated recovery systems.
- Lead Engineering Design Reviews, providing well-articulated explanations for decisions.
- Write software frameworks that can be extended by others on the team.
- Translate business requirements into software solutions.
- Know when and how to optimize complex code.
- Collaborate cross-functionally including ML engineers.
- Expert at scoping tasks.
Requirements
- 5+ years of professional software development experience.
- Deep technical subject matter expertise in 3+ general areas of software development (e.g., Big Data Processing, Distributed Systems, Data Modeling).
- 3+ years of hands-on experience in Data Engineering with a focus on supporting analytics or data science teams.
- Advanced proficiency in Python and SQL.
- Proven ability to architect and write software that enables ML at scale—moving beyond simple ETL to building robust data platforms.
- Strong background in modern data infrastructure relevant to AI (e.g., Spark, Airflow, Kafka, Vector Databases).
- Experience with Data Lake/Lakehouse architectures (e.g., Databricks, Snowflake, Delta Lake).
- Familiarity with MLOps concepts.
- Proven ability to deploy and maintain data systems in production.
Tech Stack
- Airflow
- Distributed Systems
- ETL
- Kafka
- Python
- Spark
- SQL
Benefits
- Health insurance
- 401(k) matching
- Flexible work hours
- Paid time off
- Remote work options
- Wellness and childcare subsidy
- Education discounts