Data Engineer – Senior
United States of America
Full Time
3 hours ago
$140,000 - $180,000 USD
No H1B
Key skills
PythonSQLMachine LearningData EngineeringAnalyticsDatabricksKafkadbtAWSAzureTerraformGitHub ActionsS3IAMCloudWatchGlueAthenaAzure DevOpsDatadogSplunkGitHub
About this role
BDR Solutions, LLC supports the U.S. Federal Government in achieving its mission and goals. They are seeking a Senior Data Engineer to lead cloud data engineering initiatives for the IRS, focusing on secure architecture and tenant onboarding within Databricks and AWS environments.
Responsibilities:
- Lead workspace architecture, Unity Catalog governance, and cluster policy design for IRS tenant organizations
- Perform tenant discovery and requirements gathering, source profiling, and security classification for new data intake requests
- Develop end-to-end technical designs for tenant onboarding including Data Sharing Agreements and SLA documentation
- Maintain Requirements Traceability Matrix (RTM) and prepare Architecture Review Board (ARB) submissions
- Contribute to tenant onboarding automation, financial operations integration, and ROI reporting
- Deliver tenant training materials and facilitate weekly office hours for IRS program teams
- Support P1/P2 incident response within SLA for tenant-related platform issues
- Provide NIST 800-53 control evidence within 5 business days of IRS audit requests
Requirements:
- Active MBI – mandatory; candidates in process do NOT qualify
- 8+ years in cloud data engineering; 3–5 years hands-on Databricks (Unity Catalog, Delta Live Tables, Auto Loader, Genie)
- Databricks Certified Data Engineer Professional or advanced equivalent – must be current and in good standing at award
- AWS Certified Solutions Architect or Data Engineer – Associate or Professional
- Deep experience with medallion architecture (Bronze/Silver/Gold), Delta Lake, and CDC patterns
- Proficiency in Python, PySpark, and SQL for enterprise pipeline development
- Hands-on AWS: S3, IAM, VPC, CloudWatch, Glue, Athena
- Experience with RBAC/ABAC, row-level security, PII column masking
- Familiarity with IRS OneSDLC, FISMA, NIST 800-53, and FedRAMP High
- Bachelor's degree in Computer Science, Information Technology, or related field
- Lead workspace architecture, Unity Catalog governance, and cluster policy design for IRS tenant organizations
- Perform tenant discovery and requirements gathering, source profiling, and security classification for new data intake requests
- Develop end-to-end technical designs for tenant onboarding including Data Sharing Agreements and SLA documentation
- Maintain Requirements Traceability Matrix (RTM) and prepare Architecture Review Board (ARB) submissions
- Contribute to tenant onboarding automation, financial operations integration, and ROI reporting
- Deliver tenant training materials and facilitate weekly office hours for IRS program teams
- Support P1/P2 incident response within SLA for tenant-related platform issues
- Provide NIST 800-53 control evidence within 5 business days of IRS audit requests
- U.S Citizenship is required
- Current or recent IRS contract experience strongly preferred
- Preferred Tools & Technologies: Terraform, GitHub Actions or Azure DevOps, Kafka/MSK, dbt, Great Expectations, Splunk/Datadog, Collibra/Alation, AWS Lake Formation, and experience with data mesh or modern data product architectures
- Databricks Ecosystem
- Azure DevOps or GitHub Actions
- Experience with machine learning pipelines
- Knowledge of data governance frameworks
- Familiarity with real-time streaming systems
- Certifications in cloud platforms (AWS Certified Data Analytics, Azure Data Engineer)