Data Engineer - Remote

Mindex is a software development and cloud services company with over 30 years of experience, known for delivering innovative solutions. They are seeking a Data Engineer who will be responsible for designing, developing, and optimizing data platforms to support analytics and machine learning initiatives.

Responsibilities:

• Assemble large, complex data sets that meet business requirements through extraction, transformation, and loading of data from a wide variety of data sources
• Provide operational support and troubleshooting for existing processes and systems
• Work closely with architects, solution leads, data owners, Data Scientists and key stakeholders to facilitate and coordinate the data platform backlog grooming process, triaging new feature requests in preparation for future project activities
• Deliver automation & efficient processes to ensure high quality throughput & performance of the entire data & analytics platform
• Ensure data extraction, transformation and loading data meet data security & compliance requirements
• Engage with data source platform leads to gain tactical and strategic understanding of data sources required by Agency Data Services AI/ML as well as Data Office standards
• Create data tools for data scientist team members that assist them in building and optimizing models

Requirements:

• BS degree in Computer Science, Data Science, Engineering, or equivalent software/services experience required
• 4+ years working with SQL, Snowflake, Databricks, Spark, and other big data technologies; 4+ years using Python, SQL, PySpark, R, or similar languages and manipulating, processing, and extracting value from large, disconnected data sets
• 4+ years building and optimizing data pipelines, architectures, and data sets to answer business questions and identify opportunities for improvement
• 2+ years supporting large-scale data processing and storage using Azure Data Factory, Integration Runtime, Data Lake, Databricks, Spark, Azure ML, and Cosmos DB
• 2+ years addressing privacy, compliance, and security aspects of data storage and processing; and delivering data solutions in Agile environments
• 2+ years with software development and CI/CD methodologies and tools for automated infrastructure code and MLOps and designing, implementing, and maintaining automation platforms and tools, including Ansible Tower, Azure, ARM, Terraform Enterprise, Azure DevOps, and GitHub Actions
• 2+ years with Salesforce FSC and Salesforce Data Cloud
• Strong communication and problem-solving skills
• Troubleshooting expertise
• Proficiency in Python, SQL
• Experience with Extract, Transform, Load (ETL) processes
• Experience building data pipelines
• Experience with Apache Spark and Apache Hadoop
• Experience with Amazon Web Services