Babel StreetRemote
Software Engineer III
United States of America
Full Time
4 hours ago
$110,000 - $135,000 USD
H1B Sponsor Likely
Key skills
Python 3.xScrapySeleniumPuppeteerCloudflareAkamaiPerimeterXSQLPostgreSQLMySQLHTTP/STCP/IPTLS fingerprintingBrowser-header manipulationJavaScript reverse-engineeringAPI endpoint reverse-engineeringDistributed Python programmingBrowser developer toolsRelational database design and managementJavaScriptPythonBabelPlaywrightPerformance OptimizationProblem SolvingCollaboration
About this role
Babel Street is the trusted technology partner for the world’s most advanced identity intelligence and risk operations. The Software Engineer III position focuses on architecting, developing, and maintaining automated data extraction systems to ensure high-quality data flow into internal databases.
Responsibilities:
- Spider Development: Design and deploy robust, distributed spiders and crawlers to extract data from a variety of web architectures (SPAs, SSR, etc.)
- Bot Evasion Engineering: Research and implement strategies to bypass anti-scraping technologies, including proxy rotation, browser fingerprinting, and CAPTCHA solving
- Database Management: Create and optimize SQL schemas for large-scale data storage and perform complex data transformations and validation
- System Maintenance: Proactively monitor the health of extraction agents and refactor code quickly in response to target website updates or layout changes
- Performance Optimization: Utilize asynchronous Python programming to maximize the throughput and efficiency of data collection pipelines
Requirements:
- Advanced Python: Mastery of Python 3.x with deep experience in extraction frameworks (Scrapy, Playwright, Selenium, or Puppeteer)
- Technical Resilience: Proven ability to bypass high-level bot detection (e.g., Cloudflare, Akamai, or PerimeterX)
- Database Mastery: Expert-level SQL skills and experience managing relational databases like PostgreSQL or MySQL
- Network Proficiency: Expert understanding of HTTP/S, TCP/IP, TLS fingerprinting, and browser-header manipulation
- Problem Solving: A specialized ability to reverse-engineer JavaScript-heavy websites and hidden API endpoints
- Able to write, debug, and deploy complex Python code in a distributed environment
- Must be able to analyze and interpret complex web structures and network traffic using browser developer tools
- Ability to design and maintain relational database tables containing millions of rows
- Able to pivot and respond quickly to technical 'break-fixes' to ensure data continuity for the business
- Collaboration with data analysts to define and validate data requirements and output formats
- Bachelor's degree in Computer Science, Information Systems, or a related field (or equivalent professional experience)
- Minimum of 5+ years of experience in Software Engineering, with at least 2–3 years focused specifically on large-scale web scraping or data extraction