Senior Data Engineer - Agentic AI Engineering

Boomi is a data activation company that provides infrastructure for agentic enterprises and AI-driven transformation. The Senior Data Engineer will build and operate scalable data infrastructure, pipelines, vector databases, and governance frameworks that support LLM-based agents, multi-agent workflows, training, evaluation, memory, and observability. The role also involves collaborating with AI/ML, platform, and DevOps teams to productionize experimental AI solutions and optimize them for enterprise use.

Responsibilities:

• Architect and build scalable, secure, and observable data infrastructure to power LLM-based agents, multi-agent systems, and tool-using AI workflows
• Design and operate robust batch and real-time data pipelines supporting embeddings, RAG systems, and agent memory frameworks
• Develop and manage vector database solutions to enable low-latency retrieval and contextual intelligence for AI applications
• Build data frameworks for training, evaluation, benchmarking, and continuous improvement of agentic AI systems
• Implement strong data governance, quality controls, lineage tracking, and PII/security compliance across AI data platforms
• Collaborate with AI/ML, platform, and DevOps teams to productionize experimental AI prototypes into enterprise-grade solutions
• Optimize data systems for performance, scalability, reliability, and cost efficiency across cloud environments (AWS, Azure, or GCP)

Requirements:

• 5+ years of experience building and operating large-scale data platforms as a Data Engineer
• Strong programming expertise in Python and SQL for developing scalable and efficient data solutions
• Hands-on experience designing batch and real-time data pipelines, including streaming systems like Kafka or Kinesis
• Experience with modern data platforms and cloud environments (AWS, Azure, or GCP), including tools like Snowflake
• Strong understanding of LLM/AI data workflows, including embeddings, RAG pipelines, evaluation datasets, and vector databases (Pinecone, Milvus )
• Experience with DataOps/MLOps tools such as Airflow, dbt, Lavender, and MLflow for orchestration and lifecycle management
• Strong knowledge of data quality, governance, and security, including PII handling, access controls, lineage, and ensuring data reliability
• Conshohocken, PA or Denver, CO (Preferred)
  Or US Remote
• Experience supporting agentic AI systems, multi-agent architectures, or tool-using AI agents
• Familiarity with agent memory models, conversation histories, and reasoning trace storage
• Experience with synthetic data generation and simulation frameworks
• Exposure to prompt engineering workflows and prompt/version management
• Experience optimizing data systems for low-latency AI workloads