Databricks Administrator / DBA

Databricks Administrator / DBA

Hybrid in Troy NY or Needham MA

18 Months Contract

Summary

The Senior Data Platform Engineer / DBA manages our database and data platform fleet, maintaining secure, optimized, and highly available databases, data warehouses, and lakehouses. Our stack is evolving - Redshift remains a core platform, and we are actively expanding into Databricks and open table format (Apache Iceberg on S3)

This role works closely with engineering and analytics teams to design, implement, and maintain these systems.

Responsibilities

• Operate Redshift clusters, Databricks workspaces, RDS/Aurora PostgreSQL instances, and supporting AWS infrastructure
• Perform user/security tasks across platforms: Redshift user/group management, Databricks Unity Catalog access controls, IAM Roles/Policies, RDS parameter and access management
• Design and maintain open table format data lakes using Apache Iceberg on S3, including compaction, snapshot management, partition strategies, and schema evolution
• Profile production workloads and develop strategies to run optimized clusters and workspaces with scale and efficiency
• Provide architecture guidance and support to technical leads; help develop code and SQL for data assets, identify performance tuning opportunities, and work with developers to improve production systems
• Contribute to and maintain automation scripts and internal tooling with production-quality code standards (unit tests, peer review, documentation, Git-based workflows)
• Develop monitoring and recovery automation to identify and resolve issues to meet our high SLA
• Research new technologies and develop proofs of concept; propose technical improvements and new capabilities

Requirements

• 3+ years of experience with AWS data services: Redshift, RDS/Aurora PostgreSQL, S3, EC2, Athena, Glue, Lake Formation
• 2+ years of experience with Databricks: workspace administration, Unity Catalog, cluster management, Spark SQL, Delta Live Tables or similar
• 2+ years of experience with PostgreSQL or Aurora PostgreSQL: query tuning, replication concepts, pg_stat views, parameter groups
• 2+ years of experience with AWS Security: IAM (Users, Groups, Roles, Policies, Instance Profiles), Redshift/RDS security, S3 bucket policies

Preferred

• PySpark for data pipeline development
• AWS Glue with Iceberg or Delta Lake integration
• Snowflake administration
• Terraform or AWS CDK for infrastructure as code
• DynamoDB experience
• Feature Store

Technology

• Experience with Apache Iceberg on S3: partition management, compaction, snapshot management, schema evolution
• Proficiency in Python for automation and shared tooling: object-oriented design, writing maintainable and testable code, participating in code review, and contributing to a shared Git-based codebase
• Experience automating database and platform operations using Python, Jenkins, Airflow, Datadog and Git

Soft Skills

• Ability to read, understand, debug, and extend complex Python codebases; comfort navigating large repositories with multiple modules and abstractions
• Strong analytical and critical thinking skills
• Ability to lead projects, prioritize, and multitask
• Deadline and detail-oriented
• Part of on-call rotation

Education

N/A

The estimated pay range for this position is USD $65.00/hr - USD $70.50/hr. Exact compensation and offers of employment are dependent on job-related knowledge, skills, experience, licenses or certifications, and location. We also offer comprehensive benefits. The Talent Acquisition Partner can share more details about compensation or benefits for the role during the interview process.