Key skills
Data pipeline servicesDBTSnowflakePythonSQLAWS servicesBig data platformsNoSQL databasesJavaCI/CDInfrastructure as Codecommunication skillsNoSQLMachine LearningMLETLData EngineeringData LakeAnalyticsSparkKafkaAirflowdbtHadoopCassandraAWSTerraformEKSLambdaRDSGlueKinesisAmazon KinesisCommunication
About this role
Flex is a growth-stage FinTech company headquartered in NYC, focused on creating the best rent payment experience. They are seeking a Senior Data Engineer to develop and scale their data platform, playing a pivotal role in the company's growth trajectory.
Responsibilities:
- Design, implement, and maintain high-quality data pipeline services, including but not limited to Data Lake, Kafka, Amazon Kinesis, and data access layers
- Develop robust and efficient DBT models and jobs to support analytics reporting and machine learning modeling
- Closely collaborating with the Analytics team for data modeling, reporting, and data ingestion
- Closely collaborate with the Data science team to build the data set for the ML model
- Create scalable real-time streaming pipelines and offline ETL pipelines
- Design, implement, and manage a data warehouse that provides secure access to large datasets
- Continuously improve data operations by automating manual processes, optimizing data delivery, and redesigning infrastructure for greater scalability
- Create engineering documentation for design, runbooks, and best practices
Requirements:
- A minimum of 6 years of industry experience in the data infrastructure/data engineering domain
- A minimum of 6 years of experience with Python and SQL. Java experience is a plus
- A minimum of 3 years of industry experience using DBT
- A minimum of 3 years of industry experience using Snowflake and its basic features
- Familiarity with AWS services, with industry experience using Lambda, Step Functions, Glue, RDS, EKS, DMS, EMR, etc
- Industry experience with different big data platforms and tools such as Snowflake, Kafka, Hadoop, Hive, Spark, Cassandra, Airflow, etc
- Industry experience working with relational and NoSQL databases in a production environment
- Strong fundamentals in data structures, algorithms, and design patterns
- Prior experience working on cross-functional teams
- Have the experience to build and improve user onboarding funnel and design the comprehensive experiment to drive metric improvements
- Industry experience in using Infrastructure as Code tools, specifically CDK and Terraform
- Experience with CI/CD to improve code stability and code quality
- Motivated to help other engineers succeed and be effective
- Excited to work in an ambiguous, fast-paced, and high-growth dynamic environment
- Strong written and verbal communication skills