Sr Platform Engineer, Data

Ensemble RCM LLC hires a Senior Platform Engineer, Data to build and operate reliable, secure, and scalable Azure data platform infrastructure and pipelines. The role combines hands-on data engineering, cloud platform development, networking and security, CI/CD, and operational ownership.

Responsibilities:

• Consistently deliver high-quality platform and data solutions that improve the reliability, speed, and accuracy of data reaching internal and external stakeholders
• Apply generative AI tools in day-to-day engineering work to accelerate development, improve code quality, troubleshoot complex systems, and design scalable solutions
• Design, build, test, deploy, monitor, and continuously improve scalable data pipelines using Azure Data Factory, Databricks, Apache Spark, and Python
• Implement Delta Lake patterns on Azure Data Lake Storage (ADLS) including table design, schema evolution, Z-ordering, VACUUM, and partition strategies
• Build and maintain event-driven integrations using Azure Service Bus, Event Hub, and Azure Functions to support real-time and near-real-time data flows
• Implement data quality monitoring frameworks — row counts, schema validation, null checks, freshness SLAs — as first-class pipeline components
• Develop reusable pipeline templates and platform abstractions that accelerate delivery across the data engineering team
• Translate product and business requirements into incremental, testable technical deliverables with clear acceptance criteria
• Provision and manage Azure infrastructure using infrastructure as code (Terraform, Bicep, or ARM templates), treating all platform configuration as version-controlled code
• Design and operate Azure-native services including Azure Data Factory, Azure Databricks, Azure Data Lake Storage Gen2, Microsoft Fabric (or Synapse), Azure Key Vault, and Azure Monitor
• Manage identity and access controls using Azure Active Directory, Managed Identities, RBAC, and Unity Catalog — applying least-privilege principles across all platform components
• Implement platform-level observability: pipeline SLAs, workload performance dashboards, alerting, and cost tracking via Azure Monitor, Log Analytics, and Databricks system tables
• Contribute to FinOps practices by monitoring compute costs, right-sizing clusters, configuring auto-termination policies, and flagging cost anomalies
• Support containerized workloads running on Azure Kubernetes Service (AKS) where data platform components require container-based deployment patterns
• Design and implement secure Azure networking architectures for data platform workloads — VNets, subnets, NSGs, Private Endpoints, Private DNS Zones, and service endpoints — eliminating unnecessary public internet exposure for PaaS services
• Configure, troubleshoot, and document Private Link connectivity across Azure PaaS services (ADLS, Azure SQL, Key Vault, Event Hub, Databricks); diagnose DNS resolution issues and routing problems end-to-end
• Apply encryption-in-transit and encryption-at-rest standards; maintain TLS configurations, key rotation policies, and secret management practices in line with security baselines; participate in security reviews and threat modeling for new platform components
• Partner with Cloud and Infrastructure teams on hub-and-spoke or landing zone network topologies; configure and validate firewall rules and IP allowlisting in a way that balances security posture with engineering team productivity
• Build and maintain CI/CD pipelines in Azure DevOps that automate testing, validation, and deployment of data pipelines, notebooks, and infrastructure code
• Follow and actively contribute to branching strategies, pull request standards, and code review practices; flag gaps or inconsistencies in existing workflows and propose improvements
• Write unit, integration, and data quality tests as standard deliverables — not optional additions after the feature is shipped
• Contribute to engineering standards and pattern libraries, ensuring consistency across pipelines and reducing onboarding friction for new team members
• Write well-structured, maintainable Python and SQL code; apply design patterns, modular decomposition, and API design principles appropriate to the problem at hand
• Participate in design sessions and architecture discussions, contributing hands-on perspective on what is feasible and what the operational implications of design choices will be
• Conduct thorough code and design reviews that raise the engineering bar across the team
• Identify and address technical debt within owned platform components in a planned, incremental way
• Partner with data engineering, analytics, data science, and cloud/infrastructure peers to deliver integrated platform solutions
• Provide mentorship to junior and mid-level engineers through code reviews, pair programming, and technical guidance
• Communicate platform capabilities, limitations, and trade-offs clearly to product and business stakeholders

Requirements:

• 6+ years of hands-on engineering experience, with meaningful time spent across data engineering and cloud platform or infrastructure domains
• 3+ years working with Microsoft Azure services in a production environment (ADF, ADLS Gen2, Databricks, Azure Functions, Service Bus, Event Hub, Azure Monitor)
• 2+ years of hands-on Azure networking experience: VNets, NSGs, Private Endpoints, Private DNS, service endpoints, and firewall configurations
• 2+ years developing with Apache Spark and Databricks, including Delta Lake table design, schema evolution, and performance optimization
• 4+ years of production SQL development — complex transformations, query tuning, indexing, and data modeling at enterprise scale
• Strong Python development skills applied to data engineering, pipeline orchestration, or platform tooling
• Demonstrated experience with infrastructure as code (Terraform preferred; Bicep or ARM acceptable)
• Hands-on experience building and maintaining CI/CD pipelines in Azure DevOps; familiarity with trunk-based development and pull request workflows
• Experience with event-driven architectures using Azure Service Bus, Event Hub, or equivalent messaging platforms
• Working knowledge of Azure identity and access management: Azure AD, Managed Identities, RBAC, and Key Vault
• Strong understanding of engineering fundamentals: automated testing, code reviews, telemetry, iterative delivery, and DevOps
• Bachelor's degree in Computer Science, Engineering, Information Systems, or equivalent practical experience
• Must be inquisitive and demonstrate openness to innovation including AI to explore better processes and ways to alleviate friction and improve patient and client experiences
• Due to business, operational, payroll, and regulatory requirements, this position is limited to individuals who reside and are authorized to work within the United States
• Familiarity with data platform security and governance: Unity Catalog, column-level security, data masking, and audit logging
• Background in software engineering (APIs, microservices, or distributed systems) that informs platform design and operationalization decisions
• Experience with polyglot storage architectures: relational, columnar (Parquet/Delta), key-value, or graph systems
• Exposure to healthcare datasets; familiarity with HL7 or EDI standards is a plus
• Experience supporting AI/ML or advanced analytics workloads — feature stores, training data pipelines, or model serving infrastructure
• Familiarity with AKS, container networking, and Kubernetes-based workload deployment patterns
• Azure certifications relevant to the role (e.g., AZ-305, DP-203, AZ-700) are a plus, but are not a substitute for demonstrated hands-on experience
• Familiarity with FinOps practices, Databricks cost management, and Azure cost optimization tooling