Key skills
AzureCloudETLKafkaMongoDBPySparkPythonSQLDatabricksAzure DevOpsCosmos DBMentoring
About this role
Role Overview
- Perform data ingestion, transformation, and provisioning in a Big Data ecosystem.
- Perform data cleansing and processing, analyze and assess data quality, and identify opportunities to optimize processes.
- Implement and modify data pipelines.
- Create data models.
- Extract, transform, and load (ETL) data from various sources and formats.
Requirements
- Experience with PySpark, Python, and SQL for data preparation and manipulation.
- Experience reading and writing Big Data file formats.
- Experience working on the Azure cloud platform.
- Experience with Databricks, Storage Accounts, Cosmos DB (MongoDB), Data Factory, and Azure DevOps.
- Ability to elicit and analyze business and technical requirements.
- Experience ingesting and processing structured and, especially, semi-structured data.
- Experience with data streaming projects (plus).
- Experience with Confluent Kafka (plus).
- Experience optimizing data pipeline performance (plus).
- Databricks cluster configuration experience (plus).
- Experience mentoring team members and sharing best practices and technical knowledge (plus).
- Technical English.
Tech Stack
- Azure
- Cloud
- ETL
- Kafka
- MongoDB
- PySpark
- Python
- SQL
Benefits
- Meal and grocery allowance (Flash benefits card)
- Health insurance
- Dental plan
- Life insurance
- Performance-based bonus (PPR)
- TotalPass
- Childcare allowance
- Well-being program (for physical and mental health)
- Corporate university (our #SensediaAcademy)
- Cultural and educational partners with special discounts
- Extended maternity and paternity leave
- Flexible work model.