AI Data Engineer (Hybrid)

Job Title: AI Data Engineer 
Location: Rockville, MD (Hybrid)
Duration: Long Term
 
 Role Overview 
We are seeking an AI Data Engineer to design, build, and optimize data pipelines and retrieval systems for a Generative AI platform. This role focuses on ingesting, transforming, and indexing domain-specific data to enable accurate, context-aware AI responses.

You will collaborate closely with AI/Agent developers and platform engineering teams to improve retrieval quality and expand knowledge coverage.
   
Key Responsibilities 
1. Data Engineering & ETL

• Design and develop scalable ETL pipelines for structured and unstructured data

• Build workflows for document parsing, transformation, and large-scale ingestion

• Implement data validation and quality checks to ensure accuracy and completeness

• Utilize AWS services such as S3, Lambda, Step Functions, OpenSearch, and Bedrock

  
2. RAG Pipeline Development & Search Optimization

• Architect and optimize Retrieval-Augmented Generation (RAG) pipelines

• Define document chunking strategies and generate vector embeddings

• Enhance retrieval quality through ranking, filtering, and hybrid search techniques

• Evaluate system performance using retrieval accuracy metrics and benchmarks

• Experiment with embedding models and search strategies to improve response relevance

  
3. Quality Engineering & Testing

• Design test strategies for validating data pipelines and ingestion workflows

• Develop automated regression tests to monitor retrieval performance

• Build evaluation frameworks to measure precision, recall, and relevance

• Promote Test-Driven Development (TDD) practices

  
4. Generative AI & Innovation

• Stay current with advancements in RAG, embeddings, and retrieval systems

• Explore techniques such as hybrid search, reranking, and contextual retrieval

• Collaborate with AI teams to ensure high-quality, contextually relevant outputs

  
5. Security & Compliance

• Follow secure coding practices, especially for sensitive and PII data

• Ensure compliance with organizational security standards and policies

• Participate in threat modeling and secure system design discussions

  
Required Skills & Qualifications

• Strong experience in data engineering and ETL pipeline development

• Hands-on experience with AWS services (S3, Lambda, Step Functions, OpenSearch)

• Experience with RAG pipelines, vector databases, and embeddings

• Proficiency in Python and data processing frameworks

• Understanding of search optimization, ranking, and retrieval techniques

• Experience with data validation, testing frameworks, and TDD

• Familiarity with Generative AI platforms and LLM integrations

Nice to Have

• Experience with Amazon Bedrock or similar LLM platforms

• Knowledge of hybrid search and reranking techniques

• Exposure to MLOps or AI platform engineering

• Experience handling PII and secure data systems