Sr. Data Engineer - Azure

About Simform:

Simform is a premier digital engineering company specializing in Cloud, Data, AI/ML, and Experience Engineering to create seamless digital experiences and scalable products. Simform is a strong partner for Microsoft, Google Cloud, and Databricks. With a presence in 5+ countries, Simform primarily serves North America, the UK, and the Northern European market. Simform takes pride in being one of the most reputed employers in the region, having created a thriving work culture with a high work-life balance that gives a sense of freedom and opportunity to grow.  

Role Overview:

Key Responsibilities

• Design, develop, monitor, and maintain end-to-end data pipelines on Azure.
• Work with Azure services such as Azure Data Factory (ADF), Azure Synapse, Databricks, and Azure Blob Storage for data ingestion, processing, and analytics.
• Develop and optimize data ingestion, transformation, and analytical workflows for structured and unstructured data.
• Design efficient data models to ensure optimal performance in data processing and storage.
• Implement large-scale data ingestion pipelines capable of handling TBs of data.
• Develop scalable and high-throughput distributed batch or real-time data solutions using Apache Kafka, Apache Airflow, and Databricks.
• Build and maintain ETL/ELT pipelines for data integration and migration.
• Work extensively with relational databases (PostgreSQL, MySQL, SQL Server) and NoSQL databases (MongoDB, CosmosDB, Cassandra, GraphDB).
• Optimize SQL queries, performance tuning, indexing, partitioning, and denormalization strategies.
• Collaborate with cross-functional teams, including data scientists and software engineers, to integrate data solutions into production environments.
• Ensure data quality, integrity, and compliance with security best practices.
• Participate in client interactions and stakeholder meetings to gather requirements and provide technical insights.

Required Skills & Qualifications

• Bachelor’s/Master’s degree in Computer Science, Data Engineering, or a related field.
• 5+ years of hands-on experience in data engineering, ETL development, and data pipeline implementation.
• Proficiency in Python and SQL for data processing and analysis.
• Strong expertise in Azure data services (ADF, Synapse, Databricks, Blob Storage, etc.).
• Experience with big data processing frameworks like Spark, Kafka, and Airflow.
• Working knowledge of data warehouse solutions such as Synapse, Snowflake, Redshift, or BigQuery.
• Experience with NoSQL databases (MongoDB, CosmosDB, Cassandra, GraphDB) and relational databases.
• Strong analytical and problem-solving skills.
• Ability to work independently, mentor peers, and meet tight deadlines.
• Excellent interpersonal and communication skills.
• Experience with cloud-based data architecture and security best practices.
• Azure certification is a plus.

Preferred Qualifications (Nice to Have)

• Experience with data lake architectures.
• Knowledge of machine learning model deployment and data processing for AI applications.
• Prior experience in automated data pipeline deployment using CI/CD workflows.