Hyper Lychee LabsRemote
Lead Data Engineering (Azure/Python)
United States
Full-time
2 hours ago
H1B Sponsor
Key skills
Data Engineering ExpertisePythonAzure EnvironmentCI/CD (Azure DevOps)SQLPolarsDuckDBDockerCloud SecurityConsultative MindsetDomain KnowledgeCommunicationTeam LeadershipCollaborationData EngineeringData LakeBIBusiness IntelligencePower BIAzureAzure DevOpsSalesforceCRMCI/CD
About this role
Hyper Lychee Labs is seeking a Lead Data Engineer to spearhead the architecture, development, and maintenance of their modern data lake. The role involves ensuring efficient, secure, and accurate data processing while collaborating with Business Intelligence teams and implementing a modern tech stack within the Azure ecosystem.
Responsibilities:
- Data Lake Architecture & Governance: Design and implement a scalable, governed data lake solution tailored for analytical scaling
- Pipeline Development: Build, orchestrate, and optimize high-performance data processing pipelines using Python, Polars, DuckDB, and SQL
- Containerization & Deployment: Implement and manage Docker/containers for running data jobs reliably using Azure Containers
- Security & Access Management: Drive secure development practices. Create and manage Azure roles and permissions (RBAC) to ensure strict data governance and secure environments
- Cross-Functional Collaboration: Partner closely with the BI team to understand their reporting requirements (Power BI) and deliver the data models they need
- CI/CD Implementation: Establish and maintain automated deployment pipelines using Azure DevOps
Requirements:
- 8-10 Years of experience (minimum 5-6 years working on Data Lake solutions)
- Proven track record in core Data Engineering principles, data modeling, and data warehouse/data lake architecture
- Exceptional proficiency in Python, SQL, Polars, and DuckDB
- Deep familiarity with the Azure Environment, including deploying and managing Azure Containers
- Strong practical knowledge of CI/CD pipelines (Azure DevOps) and Docker/containerization
- Solid understanding of cloud security, specifically in configuring Azure roles, identities, and access management
- Previous experience working with Fintech industry data, understanding its unique security and analytical requirements
- Excellent communication skills with the ability to translate technical concepts to BI teams and business stakeholders
- Experience managing a team of 4-5 Data Engineers
- Strong capabilities of collaborating with Senior Stakeholder, Product Owner and team
- Capabilities to provide an optimized solution and have FinTech and Wealth Management domain understanding
- Familiarity with Microsoft Dynamics, Salesforce, or similar CRM platforms
- Knowledge of Azure Data Factory (ADF) for data integration
- Experience with Microsoft Purview for unified data governance