Databricks Data Engineer

Soho Square Solutions is seeking a Databricks Data Engineer to optimize cloud costs and performance. The role involves collaborating with stakeholders to establish budgets, recommending infrastructure solutions, and analyzing cloud resource usage to minimize waste.

Responsibilities:

• Collaborate with stakeholders to understand cost requirements and establish budgets for cloud services
• Recommend and design infrastructure solutions optimized for cost savings and performance
• Set up tools and dashboards to continuously track cloud expenses, usage patterns, and budget adherence
• Analyze cloud resource usage and adjust (e.g., right-sizing, scheduling) to minimize waste and reduce costs
• Maintain records of cloud budgeting processes, cost allocation models, and optimization actions
• Define and document program processes, create training material, and conduct training to end users
• Use various project delivery methodologies (Agile/Scrum, Waterfall, and hybrid) and tailor the approach for program effectiveness
• Create project plans which consist of scope, schedule, cost, quality, organization, communication, staffing, risk and issue management
• Facilitate, attend, scribe and distribute minutes for scrum/program meetings
• Drive project execution by monitoring project status, gathering customer responses and needs, monitoring project scope and watching for potential roadblocks
• Facilitate cross-team coordination to help remove barriers experienced by team

Requirements:

• Expertise on optimizing cloud costs on Databricks
• Collaborate with stakeholders to understand cost requirements and establish budgets for cloud services
• Recommend and design infrastructure solutions optimized for cost savings and performance
• Set up tools and dashboards to continuously track cloud expenses, usage patterns, and budget adherence
• Analyze cloud resource usage and adjust (e.g., right-sizing, scheduling) to minimize waste and reduce costs
• Maintain records of cloud budgeting processes, cost allocation models, and optimization actions
• Define and document program processes, create training material, and conduct training to end users
• Use various project delivery methodologies (Agile/Scrum, Waterfall, and hybrid) and tailor the approach for program effectiveness
• Create project plans which consist of scope, schedule, cost, quality, organization, communication, staffing, risk and issue management
• Facilitate, attend, scribe and distribute minutes for scrum/program meetings
• Drive project execution by monitoring project status, gathering customer responses and needs, monitoring project scope and watching for potential roadblocks
• Facilitate cross-team coordination to help remove barriers experienced by team