Duration: Approximately 5 Months
Location: Dallas, TX
Work Arrangement: Onsite availability required (please confirm)
We are seeking a Senior Databricks Platform Engineer to design, implement, and manage scalable, enterprise-grade data platforms using Databricks. The ideal candidate will have strong hands-on experience with data engineering, cloud platforms, platform administration, solution architecture, and stakeholder engagement.
Databricks: Clusters, notebooks, Delta Lake, MLflow, Unity Catalog, workspace administration, and governance.
Cloud Platforms: Hands-on experience with AWS, Azure, or Google Cloud Platform.
Data Engineering: Apache Spark, Python, SQL, and distributed data processing.
Solution Architecture: End-to-end data architecture, ingestion, transformation, storage, and analytics.
Streaming Technologies: Structured Streaming, Kafka, AWS Kinesis, or Azure Event Hubs.
DevOps: CI/CD pipelines using Azure DevOps, GitHub Actions, Jenkins, or similar tools.
Platform Administration: Cluster configuration, performance tuning, access control, security, monitoring, and cost optimization.
Leadership: Technical design discussions, code reviews, mentoring, and stakeholder collaboration.
Methodologies: Agile development and delivery practices.
Databricks Data Engineer Professional or relevant architecture certification.
AWS, Azure, or Google Cloud Platform cloud certifications.
Experience with Tableau, Power BI, or Looker.
Knowledge of machine learning workflows and MLOps.
Experience with enterprise data modernization and cloud migration projects.
1. Solution Architecture & Design
Design end-to-end architectures using the Databricks Lakehouse Platform.
Architect scalable ETL/ELT pipelines, streaming solutions, and analytics platforms.
Define data models, storage strategies, and integration patterns.
Optimize cluster configurations, platform performance, and resource utilization.
2. Technical Leadership
Lead architecture discussions and technical design workshops.
Establish engineering best practices and reusable components.
Conduct code reviews and mentor data engineers.
3. Stakeholder Engagement
Translate business requirements into scalable technical solutions.
Prepare architecture diagrams, solution proposals, and implementation plans.
Collaborate with engineering, product, and analytics teams.
4. Data Governance, Security & Compliance
Implement Unity Catalog, data lineage, access controls, and workspace governance.
Establish monitoring, auditing, and data quality practices.
Align platform implementations with applicable security and compliance requirements.
5. Continuous Improvement
Evaluate new Databricks features and complementary technologies.
Identify opportunities for modernization, performance improvement, and cost optimization.
Improve data platform reliability, scalability, and operational efficiency.
