Senior Data Engineer - GenAI & Cloud Data Platforms:
Senior Data Engineer with Generative AI expertise to design, build, and optimize scalable cloud-native data platforms and AI-powered data solutions. The ideal candidate will have strong experience in Python, AWS Data Services, PySpark, Data Warehousing, and Modern Data Engineering practices, combined with hands-on exposure to Generative AI, Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), Vector Databases, and AI-driven data pipelines.
This role requires a combination of data engineering excellence, cloud architecture expertise, and the ability to enable enterprise AI/ML initiatives through robust and scalable data foundations.
Good understanding of sales, marketing, distribution functions for an asset management firm is needed. Good understanding of Client, account, sales, marketing performance and investment data is needed.
Preferred Qualifications:
8+ years of experience in Data Engineering and Cloud Data Platforms.
2+ years of experience working with Generative AI, LLMs, or AI Engineering solutions.
AWS Certifications such as:
o AWS Certified Data Engineer
o AWS Solutions Architect
o AWS Machine Learning Specialty
Experience with MLOps and AI Operations frameworks.
Required Technical Skills:
Data Engineering & Cloud
Strong programming expertise in Python, including data engineering frameworks and APIs.
Hands-on experience with AWS Glue, AWS Lambda, Amazon S3, AWS Step Functions, and Event-Driven Architectures.
Advanced experience with PySpark and distributed data processing frameworks.
Expertise in designing and optimizing ETL/ELT pipelines for large-scale data movement and transformation.
Strong experience with Snowflake, Amazon Redshift, or similar cloud data warehouse platforms.
Deep knowledge of SQL, including:
o Complex query development
o Data modeling
o Query optimization and performance tuning
o Data analytics and troubleshooting
Experience with data orchestration tools such as Apache Airflow, AWS MWAA, Prefect, or similar frameworks.
Strong understanding of:
o Data Warehousing concepts
o Dimensional Modeling (Star/Snowflake Schema)
o Data Lakes and Lakehouse Architectures
o Cloud-native architecture patterns
o Data Governance and Metadata Management
Generative AI & AI Engineering
Hands-on experience with Generative AI solutions and Large Language Models (LLMs) such as:
o OpenAI GPT / Anthropic Claude/Amazon Bedrock/Azure OpenAI
Experience building Retrieval-Augmented Generation (RAG) solutions.
Knowledge of Vector Databases such as:
o Pinecone / Amazon OpenSearch
Experience developing AI-powered applications using:
o LangChain /AI Agent Frameworks/ AWS AgentCore
Familiarity with:
o Prompt Engineering
o Embedding Models
o Model Evaluation and Monitoring
o AI Governance and Responsible AI Practices
Experience integrating enterprise data platforms with AI/ML and GenAI ecosystems.
DevOps & Engineering Best Practices
Key Responsibilities
