Senior Data Engineer

Hamilton Lane is seeking a Senior Data Engineer to build and maintain scalable data integration solutions using Python, Snowflake, and cloud technologies. The role delivers reliable data products for analytics and business decision-making while collaborating with technical and business teams.

Responsibilities:

• Design, develop, and maintain scalable ETL/ELT data pipelines using Python (PySpark), Snowflake, and cloud-native integration technologies
• Build reliable, efficient, and reusable data ingestion, transformation, and loading processes to support enterprise analytics and reporting needs
• Utilize Snowflake's architecture and capabilities to design, build, and optimize modern cloud data solutions
• Implement and manage Snowflake objects including databases, schemas, tables, views, streams, tasks, stages, and stored procedures
• Leverage Snowflake features such as virtual warehouses, data sharing, time travel, and automated scaling to maximize performance and cost efficiency
• Apply expertise in dimensional modeling, star schemas, facts, and dimensions to design and implement scalable enterprise data warehouse solutions within Snowflake
• Develop data models that balance business requirements, performance, and maintainability
• Extract and ingest data from a variety of sources including REST APIs, relational databases, SaaS applications, flat files, and cloud storage platforms
• Develop and maintain robust ingestion frameworks supporting structured and semi-structured data formats
• Contribute to the design and implementation of modern data platform concepts including data lakes, lakehouses, data mesh architectures, and enterprise data catalogs
• Support integration between Snowflake and cloud-native services across Azure and other cloud platforms
• Collaborate with data architects and business stakeholders to develop logical and physical data models aligned with business objectives
• Establish and enforce data engineering standards and best practices
• Implement automated data quality controls, validation frameworks, and monitoring processes to ensure accuracy, consistency, and completeness
• Support data governance initiatives and maintain adherence to organizational standards
• Monitor and optimize Snowflake workloads, ETL/ELT processes, and SQL queries to meet performance objectives and service-level agreements
• Analyze warehouse utilization and recommend improvements to performance and cost efficiency
• Monitor data pipelines and platform operations, diagnose data and performance issues, and implement long-term solutions to ensure reliability and availability
• Support production environments and participate in incident resolution activities
• Maintain comprehensive documentation for data pipelines, data flows, transformations, data models, and operational processes
• Partner with cross-functional teams to understand business requirements and provide technical expertise on data-related initiatives
• Ensure data security, privacy, and regulatory compliance through the implementation of appropriate access controls, masking policies, governance frameworks, and industry best practices

Requirements:

• Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a related field; or equivalent professional experience
• Proven experience developing data pipelines that extract, transform, and load data from REST APIs, relational databases, cloud storage, and flat-file sources
• Demonstrated experience working with Snowflake features including virtual warehouses, streams, tasks, stages, Snowpipe, secure data sharing, and performance optimization
• Advanced SQL development skills with the ability to write complex queries, tune performance, and optimize large-scale data workloads
• Experience with data modeling techniques including dimensional modeling, star schemas, fact tables, and dimension tables
• Familiarity with cloud-based data ecosystems and integration services, particularly within Microsoft Azure
• Experience managing source code and CI/CD pipelines using Git and Azure DevOps or similar platforms
• Knowledge of data integration best practices, data governance frameworks, and enterprise data management principles
• Strong analytical and problem-solving skills with exceptional attention to detail
• Excellent verbal and written communication skills and the ability to work collaboratively in a fast-pace environment with evolving priorities
• Relevant certifications such as SnowPro, Azure Data Engineer Associate, or other cloud data platform certifications are a plus
• Experience with big data technologies, machine learning, data science platforms, or advanced analytics is preferred
• Experience with data visualization tools such as Power BI, Tableau, or similar platforms is a plus
• Familiarity with Agile delivery methodologies and DevOps practices is desirable