Senior Data Engineer

ADPMN Inc is seeking a Senior Data Engineer with strong hands-on experience in Python and Apache Spark to design, develop, and optimize scalable data pipelines and distributed data processing solutions. The ideal candidate will be responsible for building production-grade data solutions in an Agile environment and collaborating with various stakeholders to deliver high-quality data solutions.

Responsibilities:

• Design, develop, and optimize enterprise data pipelines using Python and Apache Spark
• Build scalable ETL/ELT solutions for large datasets
• Design and maintain data models supporting analytics and reporting
• Optimize Spark jobs for performance, scalability, and reliability
• Develop APIs and data services supporting enterprise applications
• Implement automated testing, CI/CD, and data quality controls
• Collaborate with architects, software engineers, data scientists, and business stakeholders to deliver high-quality data solutions
• Participate in code reviews, technical design discussions, and production support

Requirements:

• 10+ years of Data Engineering or Software Engineering experience
• Strong hands-on Python experience (Required)
• Strong hands-on Apache Spark / PySpark experience (Required)
• Experience building scalable ETL/ELT pipelines
• Strong knowledge of data modeling, SQL, distributed computing, and data warehousing
• Experience with pipeline orchestration (Airflow, Databricks Workflows, or similar)
• Experience developing and consuming REST APIs
• Strong understanding of software engineering principles, including object-oriented programming, design patterns, data structures, and algorithms
• Experience with Git, CI/CD pipelines, automated testing, and Agile/Scrum
• Excellent analytical, troubleshooting, and communication skills
• AWS experience (S3, EC2, Glue, Lambda, EMR)
• Experience with Docker, Kubernetes, Delta Lake, Snowflake
• Databricks experience and/or Apache Spark certification
• Experience with R or SAS is a plus