Sr. Data Engineer - Fabric (100% Remote)

ClearCaptions, LLC is dedicated to improving the lives of seniors and their caregivers through innovative communication technologies. The Sr. Data Engineer – Fabric will lead the design and implementation of enterprise-scale data solutions using Microsoft Fabric, collaborating with various teams to ensure reliable and scalable data operations while mentoring other data engineers.

Responsibilities:

• Collaborates with cross-functional teams to understand data and analytical requirements and translates them into effective Microsoft Fabric–based data solutions
• Mentors and provides technical leadership to Data Engineers through code reviews, design reviews, knowledge-sharing sessions, and engineering best practices
• Establishes and maintains data engineering standards, reusable frameworks, development patterns, and operational best practices to improve consistency and scalability across the data platform
• Leads root cause analysis and resolution efforts for critical production issues impacting enterprise data pipelines, reporting solutions, and platform operations
• Reviews and provides guidance on solution designs, architecture decisions, and implementation approaches to ensure alignment with enterprise standards and best practices
• Designs, develops, and maintains end-to-end data ingestion and transformation processes using Microsoft Fabric OneLake, Lakehouses, Data Pipelines, Dataflows Gen2, and Notebooks
• Formats, cleanses, and stores data in a structured manner to ensure data quality and accessibility for reporting and analysis, following Medallion Architecture
• Works closely with reporting and analytics teams to ensure datasets are accurate, well-modeled, and optimized for Power BI and other analytical workloads
• Creates comprehensive documentation for data pipelines, processes, and solutions
• Participates in data governance activities to ensure data integrity, security, and compliance
• Utilizes source control to manage and track changes to data pipelines and codebase
• Provides level II/III support to troubleshoot and resolve data-related issues and inquiries
• Provides ancillary support for Machine Learning (ML) and Artificial Intelligence (AI) processes
• Evaluates emerging technologies, industry trends, and architectural patterns related to data engineering, analytics, artificial intelligence, and cloud platforms and provides recommendations for adoption
• Manages and optimizes Microsoft Fabric platform performance by monitoring capacity utilization, query performance, Lakehouse and Warehouse workloads, Delta table maintenance (OPTIMIZE/VACUUM), data partitioning, storage consumption, and pipeline execution to ensure scalable, reliable, and cost-effective data operations
• Develops and maintains monitoring, alerting, and observability solutions for Microsoft Fabric pipelines, notebooks, Lakehouses, Warehouses, semantic models, and related data platform components to ensure operational reliability and rapid issue resolution
• Performs other duties as assigned

Requirements:

• Bachelor's degree in computer science, information technology, or a related field (or equivalent experience)
• A minimum of six (6) years of proven experience in data engineering, ETL development, and data warehousing
• Demonstrated experience leading complex data engineering projects from architecture and design through implementation and operational support
• Demonstrated ability to lead through influence, collaboration, and technical expertise, including mentoring Data Engineers and cross-functional technical teams, leading initiatives and workstreams, and driving successful cross-functional outcomes
• Experience establishing engineering standards, governance processes, development frameworks, and operational best practices
• Advanced troubleshooting and performance optimization experience across cloud data platforms, distributed processing frameworks, and enterprise data solutions
• Strong hands-on experience with Microsoft Fabric, including OneLake, Lakehouses, Data Pipelines, Dataflows Gen2, and/or Notebooks
• Solid understanding of Medallion Architecture, Delta Lake concepts, data modeling, and modern analytics patterns
• Experience with data governance, data quality, and data security frameworks and standards
• Solid grasp of Continuous Integration Continuous Delivery (“CI/CD”), including source control, deployment pipelines, and release management practices
• Experience with Git-based source control and collaborative development workflows
• Expertise in integrating Fabric solutions with Power BI, Azure services, and external data sources
• Experience monitoring and tuning cloud-based data platforms for performance, scalability, and cost optimization
• Experience with Apache Spark and PySpark within Microsoft Fabric
• Excellent verbal and written communication skills, presentation, problem-solving skills and attention to detail
• Self-starter with strong organizational and time management skills, self-directed and able to handle multiple priorities with demanding timeframes
• Ability to work independently and collaboratively with colleagues and staff to create a high-quality, results-driven, team-oriented, fast-paced environment
• Demonstrated ability to use discretion, make sound decisions, and maintain confidentiality
• Willingness and ability to work flexible hours
• Proficiency in MS Office (including Excel, Outlook, PowerPoint, Visio, & Word) and modern communication tools for virtual teams (e.g., MS Teams, Slack)
• Expertise in developing data pipelines to support Machine Learning (ML) and / or Artificial Intelligence (AI) tools is a plus