Key skills
AirflowAzureCloudETLKafkaKubernetesOraclePythonSQLAILLMRAGELTAnalyticsSnowflakeDatabricks
About this role
Role Overview
- Build and maintain end-to-end data ingestion and ETL pipelines
- Ingest structured and unstructured data from APIs, Snowflake, Oracle, documents, and market data providers
- Develop production-quality Python services and data workflows
- Orchestrate pipelines using Kubernetes and workflow tools
- Apply clean architecture and strong engineering design patterns
- Ensure data quality, reliability, and security
- Design schemas and transformation layers for analytics and AI agents
- Prepare data for AI and RAG systems (chunking, embeddings, retrieval pipelines)
Requirements
- Strong Python (advanced, production-grade)
- Advanced SQL
- ETL/ELT pipeline design and orchestration (Airflow, ADF, Databricks or similar)
- Experience with RAG systems, embeddings, and vector databases
- Hands-on experience with Cloud providers (Microsoft Azure desired)
- Document parsing and unstructured data processing
- Experience with data lakehouse architectures
- API and database integrations
- Kubernetes-based workloads
- Nice to Have Experience:
- Experience with Snowflake and Oracle
- Streaming systems (Kafka, Event Hubs)
- Data Lakehouse architectures
- LLM evaluation and prompt engineering
- Experience building AI or data platforms.
Tech Stack
- Airflow
- Azure
- Cloud
- ETL
- Kafka
- Kubernetes
- Oracle
- Python
- SQL
Benefits
- The chance to join an organization with triple-digit growth that is changing the paradigm on how software products are built.
- The opportunity to form part of an amazing, multicultural community of tech experts.
- A highly competitive compensation package.
- Medical insurance.