Senior Data Engineer

Hamilton Lane is seeking a Senior Data Engineer to design, develop, and maintain scalable data integration solutions using Python (PySpark), Snowflake, and modern cloud data technologies. The role supports reliable data products for analytics, reporting, and operational decision-making in collaboration with technical and business teams.

Responsibilities:

• Design, develop, and maintain scalable ETL/ELT data pipelines using Python (PySpark), Snowflake, and cloud-native integration technologies
• Build reliable, efficient, and reusable data ingestion, transformation, and loading processes to support enterprise analytics and reporting needs
• Utilize Snowflake's architecture and capabilities to design, build, and optimize modern cloud data solutions
• Implement and manage Snowflake objects including databases, schemas, tables, views, streams, tasks, stages, and stored procedures
• Leverage Snowflake features such as virtual warehouses, data sharing, time travel, and automated scaling to maximize performance and cost efficiency
• Apply expertise in dimensional modeling, star schemas, facts, and dimensions to design and implement scalable enterprise data warehouse solutions within Snowflake
• Develop data models that balance business requirements, performance, and maintainability
• Extract and ingest data from a variety of sources including REST APIs, relational databases, SaaS applications, flat files, and cloud storage platforms
• Develop and maintain robust ingestion frameworks supporting structured and semi-structured data formats
• Contribute to the design and implementation of modern data platform concepts including data lakes, lakehouses, data mesh architectures, and enterprise data catalogs
• Support integration between Snowflake and cloud-native services across Azure and other cloud platforms
• Collaborate with data architects and business stakeholders to develop logical and physical data models aligned with business objectives
• Establish and enforce data engineering standards and best practices
• Implement automated data quality controls, validation frameworks, and monitoring processes to ensure accuracy, consistency, and completeness
• Support data governance initiatives and maintain adherence to organizational standards
• Monitor and optimize Snowflake workloads, ETL/ELT processes, and SQL queries to meet performance objectives and service-level agreements
• Analyze warehouse utilization and recommend improvements to performance and cost efficiency
• Monitor data pipelines and platform operations, diagnose data and performance issues, and implement long-term solutions to ensure reliability and availability
• Support production environments and participate in incident resolution activities
• Maintain comprehensive documentation for data pipelines, data flows, transformations, data models, and operational processes
• Partner with cross-functional teams to understand business requirements and provide technical expertise on data-related initiatives
• Ensure data security, privacy, and regulatory compliance through the implementation of appropriate access controls, masking policies, governance frameworks, and industry best practices

Requirements:

• Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a related field; or equivalent professional experience
• Proven experience developing data pipelines that extract, transform, and load data from REST APIs, relational databases, cloud storage, and flat-file sources
• Demonstrated experience working with Snowflake features including virtual warehouses, streams, tasks, stages, Snowpipe, secure data sharing, and performance optimization
• Advanced SQL development skills with the ability to write complex queries, tune performance, and optimize large-scale data workloads
• Experience with data modeling techniques including dimensional modeling, star schemas, fact tables, and dimension tables
• Familiarity with cloud-based data ecosystems and integration services, particularly within Microsoft Azure
• Experience managing source code and CI/CD pipelines using Git and Azure DevOps or similar platforms
• Knowledge of data integration best practices, data governance frameworks, and enterprise data management principles
• Strong analytical and problem-solving skills with exceptional attention to detail
• Excellent verbal and written communication skills and the ability to work collaboratively in a fast-pace environment with evolving priorities
• Relevant certifications such as SnowPro, Azure Data Engineer Associate, or other cloud data platform certifications are a plus
• Experience with big data technologies, machine learning, data science platforms, or advanced analytics is preferred
• Experience with data visualization tools such as Power BI, Tableau, or similar platforms is a plus
• Familiarity with Agile delivery methodologies and DevOps practices is desirable