Sr. AI Data Engineer

Grace Hill provides industry-leading SaaS technology solutions designed to improve the lives of people where they work and live. The Senior AI Data Engineer will own the data acquisition, serving, and reporting infrastructure, ensuring that systems are reliable and automated while utilizing AI technologies.

Responsibilities:

• Own and self-heal our fleet of web scrapers—build LLM-assisted resilience so structural, markup, and anti-bot changes are detected, diagnosed, and self-repaired with minimal manual effort. When something does break, agents do the first pass on root-cause and propose fixes; you review and approve
• Keep daily scraping runs stable—monitoring, alerting, retries, and graceful handling of upstream failures so data lands reliably each morning
• Use LLMs for resilient parsing and entity extraction from messy or changing HTML, reducing reliance on brittle selectors
• Own and optimize the serving layer and the ETL/ELT pipelines feeding our BigQuery warehouse—ensuring data is fresh, performant, and reliable for live use
• Build our reporting infrastructure—data models, transformations, and dashboards—plus AI-native layers like natural-language query and LLM-generated narrative insight
• Drive data quality through both rule-based checks and ML/LLM-based anomaly detection, and manage anti-bot challenges (proxies, rate limiting, request patterns) within legal and ethical guidelines
• Build and maintain production-grade MCP servers and agentic workflows that expose our data and tooling to internal and AI consumers
• Partner with the Principal Engineer, analysts, product, and leadership; document systems and best practices for maintainability and human-in-the-loop AI operations

Requirements:

• 6+ years in data engineering, including ownership of mission-critical production systems
• Strong Python with deep experience building, maintaining, and debugging scrapers (e.g., Scrapy, Playwright, Selenium, BeautifulSoup)
• AI-first: Hands-on experience building LLM-powered and agentic workflows in production—not just calling an API, but designing systems where agents do meaningful work under human supervision—including production-grade MCP servers
• Prompt engineering and LLM evaluation/observability—reasoning about output quality, cost, latency, and failure modes the way you'd reason about uptime—plus fluency with AI-assisted dev tools (e.g., Claude Code, Cursor)
• Proven experience designing reporting/analytics layers—data modeling, transformations (e.g., dbt), and BI tools
• Hands-on with the GCP data stack—BigQuery, Cloud Composer (managed Airflow), Cloud Storage, Cloud Run or GKE—plus advanced SQL and Docker
• A reliability mindset—proven track record owning systems, triaging failures, and being accountable for uptime; sound judgment on when to use deterministic code versus an LLM
• Understanding of the legal and ethical considerations around web scraping
• Experience training, deploying, and maintaining ML models
• Experience with MotherDuck / DuckDB, ideally serving data to production applications
• Experience scaling or refactoring distributed scraping systems
• Knowledge of Pub/Sub, Dataflow, or other large-scale data processing tools
• Infrastructure-as-code (Terraform)
• Experience setting data strategy or mentoring other engineers