Azure Data Engineer
Dallas,TX/Phoenix,AZ locals Only
W2 preferred
Face to face interview mandatory for Final round
Key Responsibilities
Design, develop, and maintain scalable data pipelines using Azure Databricks and Python.
Build and optimize ETL/ELT workflows for structured and unstructured data.
Develop high-performance data processing solutions using Py Spark and Spark SQL.
Design and implement data lake architecture leveraging Delta Lake.
Create and manage data ingestion frameworks from multiple enterprise sources.
Implement data quality, validation, governance, and monitoring mechanisms.
Must have hands on experience using Gen AI tools like GitHub CoPilot
Optimize Databricks jobs, clusters, notebooks, and workflows for performance and cost efficiency.
Collaborate with business analysts, data scientists, architects, and application teams to deliver enterprise data solutions.
Support batch and near real-time data processing requirements.
Participate in architecture discussions, code reviews, and best-practice implementation.
Ensure secure, scalable, and reliable cloud data solutions aligned with organizational standards.
Required Technical Skills
Primary Skills (Must Have)
Azure Databricks
Python
Py Spark
Delta Lake
Spark SQL
Data Engineering
ETL/ELT Development
Azure Data Lake Storage (ADLS Gen2)
SQL
Azure Platform Skills
Azure Databricks
Azure Data Factory (ADF)
Azure Storage Accounts
Azure Key Vault
Azure Monitor
Azure App Insights
Azure Networking Concepts
Secondary Skills
Data Modeling (Star Schema, Snowflake Schema)
DataWarehousing Concepts
Performance Tuning and Optimization
REST API Integration
NoSQL Databases
Azure Synapse Analytics
Advanced Python development (building production grade services, APIs, or data platforms)
Cloud native development Azure Databricks
Strong experience with software design, testing, and CI/CD pipelines
PostgreSQL, Kubernetes, Redis, Git, Github Actions, Maven, Junit, Agile.
