Senior Data Engineer

Ad Hoc is a technology company that delivers scalable digital services for government organizations. The Senior Data Engineer will design, build, and optimize secure, reliable, and scalable data platforms and pipelines while contributing to data strategy, architectural decisions, process improvements, and technical leadership.

Responsibilities:

• Evaluate and recommend multiple technical approaches to solve complex data engineering and architecture challenges
• Design, develop, maintain, and optimize scalable data pipelines supporting both batch and real-time processing
• Design and implement robust ETL/ELT workflows that transform raw data into reliable, consumable datasets
• Build and maintain scalable data models, data warehouses, and cloud-native data architectures
• Develop solutions for structured, semi-structured, and unstructured data sources
• Generate data architecture recommendations and successfully implement approved solutions
• Ensure data quality, integrity, governance, lineage, security, and observability across data platforms
• Optimize database performance, query execution, storage strategies, and overall system scalability
• Diagnose and resolve production issues while implementing long-term improvements to increase system reliability and performance
• Collaborate with cross-functional teams to translate business requirements into scalable technical solutions
• Present technical designs, architecture diagrams, and implementation strategies to clients, stakeholders, partners, and engineering teams
• Champion data engineering best practices, coding standards, automation, and operational excellence
• Mentor junior engineers through technical guidance, code reviews, design discussions, and knowledge sharing
• Lead small projects or serve as the technical lead for data engineering initiatives when needed
• Effectively communicate technical challenges, risks, and progress with engineering teams, leadership, clients, and stakeholders
• Participate in technical interviews and contribute to hiring decisions

Requirements:

• • Bachelor's degree in Computer Science, Information Systems, Engineering, Mathematics, or a related technical discipline with 7+ years of professional experience. Relevant years of experience may be substituted for formal education
• • 7+ years of experience designing, building, and maintaining enterprise-scale data platforms and data pipelines
• • Strong proficiency in SQL with experience developing, optimizing, and troubleshooting complex queries and large datasets
• • Professional experience developing software and data solutions using Python, Java, Scala, or a comparable programming language
• • Experience designing, developing, and maintaining ETL/ELT pipelines for batch and real-time data processing
• • Experience working with cloud-based data platforms and services within AWS, Microsoft Azure, or Google Cloud Platform
• • Experience with modern cloud data platforms such as Snowflake, Databricks, Amazon Redshift, Google BigQuery, or Azure Synapse Analytics
• • Experience with relational database technologies such as PostgreSQL, SQL Server, Oracle, or MySQL, along with familiarity with NoSQL database solutions
• • Experience with distributed data processing frameworks such as Apache Spark or equivalent big data technologies
• • Strong understanding of data modeling, dimensional modeling, schema design, data warehousing, and database optimization
• • Experience implementing data quality, validation, governance, metadata management, and data lineage best practices
• • Experience working with structured, semi-structured, and unstructured data from multiple sources
• • Experience with version control systems such as Git and CI/CD practices supporting data engineering workflows
• • Understanding of data security, privacy, encryption, and access control principles
• • Experience monitoring, troubleshooting, and optimizing production data systems for scalability, availability, and performance
• • Experience working within Agile software development environments and collaborating across cross-functional engineering teams
• • Strong analytical, troubleshooting, and problem-solving skills with the ability to make sound technical decisions
• • Excellent written and verbal communication skills with the ability to explain technical concepts to both technical and non-technical audiences
• • Demonstrated ability to mentor junior engineers through code reviews, technical guidance, and knowledge sharing
• • Ability to obtain and maintain a U.S. Public Trust clearance
• • Experience supporting U.S. Federal Government programs or other highly regulated environments
• • Experience with orchestration platforms such as Apache Airflow, Prefect, Azure Data Factory, or AWS Step Functions
• • Experience with streaming technologies such as Apache Kafka, Amazon Kinesis, or Azure Event Hubs
• • Experience using dbt or other modern data transformation frameworks
• • Experience with Infrastructure as Code tools such as Terraform or AWS CloudFormation
• • Experience working with Docker and Kubernetes in cloud-native environments
• • Experience building analytics and business intelligence solutions using Tableau, Power BI, Looker, or similar visualization tools
• • Existing Public Trust or higher security clearance
• • Experience serving as a technical lead or mentoring engineers on enterprise-scale data initiatives