Key skills
PythonSQLNoSQLAIMLLLMLangGraphChromaPineconeWeaviateETLData EngineeringKafkaKinesisPub/SubCommunicationCollaboration
About this role
Dice is the leading career destination for tech experts at every stage of their careers, and they are seeking a versatile Data Engineer who can work across traditional data engineering pipelines and next generation AI/agent based automation systems. The role involves designing, building, and maintaining ETL workflows, implementing agent-based chat automation pipelines, and ensuring data quality and operational efficiency across all pipelines.
Responsibilities:
- Design, build, and maintain traditional ETL workflows using modern data engineering tools
- Work with data streaming technologies to support real-time and near real-time data flows
- Implement agent-based chat/automation pipelines leveraging frameworks such as LangGraph
- Integrate and manage vector databases for AI/ML workloads
- Collaborate with stakeholders to define requirements and translate them into scalable data solutions
- Support multiple project components, including:
- Product enrichment pipeline
- Rewards agent
- Survey agent
- Additional MMC-related agent workflows
- Ensure data quality, reliability, and operational efficiency across all pipelines
Requirements:
- Hands-on experience with LangGraph and LangFuse
- Python
- Strong understanding of vector databases (e.g., Pinecone, Chroma, Weaviate)
- Knowledge of Google ADK or similar cloud-based developer kits
- Solid background in data engineering, ETL processes, and pipeline orchestration
- Exposure to data streaming (Kafka, Pub/Sub, Kinesis, etc.)
- Strong general database knowledge (SQL + NoSQL)
- Familiarity with building or integrating AI agents and automated chat-based workflows
- Strong communication and cross-functional collaboration
- Ability to work in an ambiguous, rapidly evolving environment
- Problem-solving mindset with attention to scalability and automation
- Experience with plug and play ETL tools
- Understanding of LLM frameworks and retrieval-augmented generation (RAG)
- Ability to work in fast-moving projects that blend traditional and AI-driven data engineering