Principal Data Engineer

Capital Technology Group provides expert consulting services in software development and data analytics. They are seeking a Principal Data Engineer to lead teams in evaluating new technologies, build scalable data systems, and mentor junior engineers.

Responsibilities:

• Design, build, and maintain scalable, efficient data pipelines and systems following modern data engineering best practices
• Evaluate and prototype new tools and technologies, assessing their risks and benefits
• Build and maintain scalable data pipelines and models using Databricks, dbt, and SQL (PostgreSQL)
• Develop robust data solutions leveraging AWS services (Redshift, RDS, S3, Glue, Lambda, DMS, CloudWatch)
• Implement data transformations and integrations using Python and Java
• Ensure reliability, performance, and maintainability of end-to-end data systems
• Implement infrastructure-as-code using Terraform for managing cloud resources
• Monitor and optimize system health and performance using CloudWatch and Splunk
• Mentor junior engineers and code reviews
• Clearly communicate technical topics to both technical and non-technical audiences
• Collaborate with cross-functional stakeholders to define data requirements and deliver solutions in agile environments

Requirements:

• Bachelor's degree in Computer Science, Engineering, or a related technical field
• 10+ years of professional experience in data engineering or related domains
• Strong hands-on experience with: Databricks (deep expertise required), AWS native, Dbt, SQL (PostgreSQL), Python, Java, AWS (Redshift, RDS, S3, Glue, Lambda, DMS, CloudWatch), Data Pipelines, Data Modeling, System Maintenance, Data Integration, Performance Optimization
• Proven leadership experience in technical teams
• Strong analytical and problem-solving skills
• Experience working in agile, iterative software development environments
• Ability to quickly learn and apply new technologies and domain knowledge
• Excellent written and verbal communication skills, with the ability to explain complex topics to diverse audiences
• Applicants MUST BE US Citizens and be able to obtain Public Trust clearance
• Experience with Kafka (streaming/data pipelines)
• Experience with Docker and Kubernetes for containerization and orchestration
• Proficiency with Splunk for log aggregation and system monitoring
• Experience using Terraform for infrastructure automation and management
• Strong SQL skills, including performance tuning and complex query design