Senior Data Engineer

Arcadia is the global utility data and energy solutions platform. They are seeking a Senior Data Engineer to create clean, curated, and organized data sets used across the company, focusing on building data assets that enable analysis and drive business decisions.

Responsibilities:

• Design, build and maintain the tooling that the wider Data team (and Arcadia as a whole) uses to interact with our data platform, including CI/CD pipelines for our data lakehouse, and unit/integration/validation testing frameworks for our data pipelines
• Optimize and tune data pipelines for improved performance, scalability, and reliability, and proactively monitor pipelines to address any issues or bottlenecks
• Collaborate with subject matter experts, engineers, and product managers to identify the most elegant and effective data structures to understand our constantly growing and evolving business
• Transform, test, deploy, and document data to deliver clean and trustworthy data for analysis to end-users
• Help bring engineering best practices (reliability, modularity, test coverage, documentation) to our DAG and to our Data team generally
• Collaborate with data engineers to build robust, tested, scalable and observable ELT pipelines
• Identify and implement best practices for data ingestion, transformation, and storage to ensure data integrity and accuracy
• On a daily basis, this role will perform: Data modeling: model raw data into clean, tested, and reusable datasets to represent our key business data concepts. Define the rules and requirements for the formats and attributes of data
• Data transformation: build our data lakehouse by transforming raw data into meaningful, useful data elements through joining, filtering, and aggregating source data
• Data documentation: create and maintain data documentation including data definitions and understandable data descriptions to enable broad-scale understanding of the use of data
• Employ software engineering best practices to write code and coach analysts and data scientists to do the same

Requirements:

• 4+ years as a Senior Data Engineer or Software Engineer building production data infrastructure. dbt experience is highly desirable
• 6+ years, cumulatively, in the data space (data engineering, data science, analytics, or similar)
• Expert-level understanding of conceptual data modeling and data mart design
• Proficiency in SQL and strong Python skills, especially in the context of data orchestration
• Deep experience building data pipelines and database management including Snowflake or similar, along with familiarity with data integration patterns and ELT/ETL processes
• Experience with orchestration tools like Prefect, Airflow, or Argo
• Proven ability to establish and enforce AI-related engineering best practices (e.g., security, responsible code review, and prompt discipline) within data pipelines and codebase contributions, ensuring data integrity, architectural stability, and the safe use of AI-assisted tools
• Ability to bring a customer-oriented and empathetic approach to understanding how data is used to drive the business
• Experience in technical leadership or mentorship
• Strong communication and collaboration skills
• The ability to work East Coast business hours to maximize overlap with team members in India
• Proven ability to solve complex problems in a dynamic and evolving environment
• Graduate degree in math, statistics, engineering, computer science, or related technical field
• Experience in predictive modeling and statistical analysis
• Experience with BI platforms
• Experience working in global, distributed teams
• Experience in the energy sector