Data Engineer

Job Description:

We are seeking an experienced Data Engineer to design, develop, and maintain scalable data pipelines and data processing solutions. The ideal candidate will have strong experience with cloud platforms, ETL/ELT, data warehousing, and distributed data processing.

Responsibilities:

• Design, develop, and maintain robust ETL/ELT data pipelines for batch and real-time data processing.
• Develop data engineering solutions using Python, SQL, Spark/PySpark, and related technologies.
• Work with AWS, Azure, or Google Cloud Platform cloud services to build scalable data platforms.
• Develop and optimize data lakes, data warehouses, and data pipelines.
• Integrate data from databases, APIs, flat files, and streaming platforms.
• Work with Kafka or other messaging/streaming technologies for real-time data ingestion.
• Perform data transformation, cleansing, validation, and quality checks.
• Optimize SQL queries, data pipelines, and processing jobs for performance and scalability.
• Implement data security, governance, monitoring, and reliability best practices.
• Collaborate with Data Scientists, BI Developers, Software Engineers, and business stakeholders.

Required Skills:

• 8+ years of hands-on Data Engineering experience.
• Strong Python and SQL programming skills.
• Experience with Apache Spark / PySpark.
• Strong understanding of ETL/ELT, data modeling, and data warehousing.
• Experience with AWS/Azure/Google Cloud Platform cloud platforms.
• Experience with Kafka or real-time streaming technologies.
• Knowledge of Data Lakes, Databricks, Snowflake, Redshift, BigQuery, or similar platforms.
• Experience with Git, CI/CD, Docker, and Kubernetes is a plus.
• Strong analytical, problem-solving, and communication skills.