Lead Big Data Engineer - PySpark

Logic20/20, Inc. is a ten-time 'Best Company to Work For' that fosters a collaborative and supportive environment. As a Lead Developer in the AI & Analytics practice, you will be responsible for delivering client value and ensuring high client satisfaction through effective data assessment and innovative technology solutions.

Responsibilities:

• Perform data assessment, analysis, and data model design
• Develop CI/CD pipelines and other DataOps fundamentals
• Communicate at the appropriate level to both business and technology teams to understand business needs and pain points
• Be creative in meeting the client’s core needs with their technology
• Explain technical benefits and deficits to non-technical audiences

Requirements:

• 5-10+ years of data engineering experience using Python and Spark (PySpark)
• 2+ years of experience leading the design and development of cloud ETL and data pipelines
• Experience working with fast streaming data
• Experience with Flink, Kafka, and Kubernetes
• Experience developing CI/CD pipelines
• Experience with Big Data Technologies (Hadoop, Spark/PySpark, MongoDB)
• Experience working with Azure, AWS, and/or GCP
• Demonstrated ability to identify business and technical impacts of user requirements and incorporate them into the project schedule
• Ability to travel at least once per quarter
• Extensive experience working with Palantir Foundry
• Java experience is highly preferred
• Consulting experience and/or experience working within the Utilities industry is highly preferred