ConstructorRemote
Senior Data Engineer: Data Lake
United States of America
Full Time
4 weeks ago
$80,000 - $120,000 USD
H1B Sponsor Likely
Key skills
PythonSparkTrinoRedshiftAWSTerraformCloudFormationClickHouseData Quality FrameworkData PipelinesStreaming Data ProcessingBatch Data ProcessingFastAPICubeS3DeltaAWS LambdaKinesisPrometheusOpenTelemetryPagerDutySentryFluent EnglishData Lake
About this role
Constructor is the next-generation platform for search and discovery in ecommerce, built to optimize metrics like revenue and conversion rate. They are seeking a Senior Data Engineer to work on their Data Lake Team, focusing on building and operating data platform components and improving platform reliability and usability.
Responsibilities:
- Maintain data pipeline job framework
- Develop Data Quality framework ( internal set of tools for internal and external data sources validation )
- Maintain and develop public facing data ingestion service with 17 000+ RPS
- Maintain and develop core data pipelines in batch and streaming manners
- Be a last line of support for our internal platform users
- Take a part in an on-call rotation for data platform incidents (shared across the team)
Requirements:
- Fluent English
- 4+ years building production services and data pipelines (batch and/or streaming)
- Strong experience with Python or the readiness to ramp up quickly
- Hands-on experience with at least one MPP system (Spark, Trino, Redshift etc.)
- Hands-on experience operating services in a cloud environment (AWS preferred)
- Terraform/CloudFormation or other IaC tools
- ClickHouse or similar analytical databases
- Experiences with data quality/observability tools