Data Engineer, Web Scraping

10a Labs is the safety and threat-intelligence layer trusted by frontier AI labs and Fortune 10 companies. In this role, you will design and optimize data pipelines for web scraping and processing data, collaborating with engineers to deliver insights and tools.

Responsibilities:

• Design, implement, and optimize end-to-end data pipelines for scraping and processing structured and unstructured data using Google Cloud Platform (or similar) and best practices
• Conduct ad hoc web scraping and data collection to support research and intelligence initiatives
• Prepare data for further analysis, including data cleaning, transformation, anonymization, and masking
• Contribute to the development of internal and external APIs, following best practices
• Collaborate with ML engineers, other data engineers, and software developers to deliver actionable insights and functional tools, including internal and external dashboards, APIs, and data dumps; and
• Drive other critical initiatives

Requirements:

• Degree (or equivalent work experience) in Computer Science, Engineering, Information Science, Data Science or a related field
• 2+ years of professional experience in data engineering or a closely related field
• Ability to communicate complex technical ideas clearly to non-technical audiences
• Proficiency in Python, SQL
• Experience with web scraping/crawling (e.g., Beautiful Soup, Selenium, Scrapy)
• Experience with Google Cloud Platform (or similar), including storage and database services (e.g., Cloud Storage, CloudSQL, Cloud Spanner) and workflow orchestration (e.g., Cloud Composer/Airflow, Cloud Run, Pub/Sub)
• Experience building and managing data pipelines, especially for text data
• Comfort working in fast-moving, high-impact environments, such as startups, AI research labs, or security-focused teams
• graduate degree preferred