Key skills
Distributed systemsData processing technologiesDBTAirflowWeb frameworksRailsSQLData platformsBig data technologiesSparkHadoopFlinkHiveKafkaStreaming data pipelinesBatching data pipelinesDistributed system designProgrammingAlgorithmsAnalytical skillsdbt
About this role
Instacart is transforming the grocery industry by providing essential delivery services and innovative solutions for customer needs. The Senior Software Engineer in the Data Platform team will design, develop, and maintain access controls and governance frameworks to enhance data integrity and privacy, while collaborating with various stakeholders to improve the data infrastructure.
Responsibilities:
- Designing, developing, and sustaining the comprehensive access controls and governance systems that improve the integrity and privacy of our data
- Ensuring the reliability, scalability, and security of the data platform
- Collaborating with various stakeholders and actively involve in the data infrastructure
Requirements:
- Has 5+ years of experience in software engineering
- Exhibits an in-depth understanding of distributed systems, with proven experience with data processing technologies such as DBT and Airflow, and common web frameworks such as Rails
- Proficiently uses SQL for writing and reviewing complex queries for data analysis and debugging
- Can design for scale with the entire system in mind
- Capably communicates and is comfortable seeking and receiving feedback
- Possesses strong analytical and debugging skills
- Takes a strong sense of ownership while working with large codebases and diverse suite of products
- Embraces a collaborative mindset to partner with engineers, designers, and PMs from multiple teams to co-create impactful solutions while supporting system contributions
- Communicates clearly, presents ideas well, and can influence key stakeholders at manager, director, and VP levels
- Holds a Bachelor's degree in Computer Science, Software Engineering, or a related field, or can demonstrate equivalent industry experience (4+ years)
- Have prior work experience in Data Platforms
- Hold experience with big data technologies such as Spark, Hadoop, Flink, Hive, or Kafka, and with both streaming and batching data pipelines
- Have proven experience with distributed system designs
- Possess strong general programming and algorithm skills
- Show strong attention to detail and accuracy in your implementation
- Have a strong experience writing complex and optimized SQL queries
- Appreciate a data-driven mindset