Senior, Software Engineer - Data Ingestion

Torc Robotics is a leader in autonomous driving technology, focused on developing software for automated trucks. They are seeking a Senior Software Engineer to build and support data ingestion systems and scalable pipelines for machine learning and analytics, collaborating closely with various engineering teams.

Responsibilities:

• Create robust and resilient pipelines to process massive daily volumes of data created by vehicle fleets and simulation environments
• Build and support scalable pipelines as part of Torc’s Data Factory to deliver data for ML training at scale
• Scale Torc’s data lake through a distributed storage system, data crawling and discovery
• Promote and protect the integrity of data through validation, versioning, data provenance, and governance
• Support the expansion of Torc’s data lake through acquisition of additional data sets from internal and external sources
• Assist in the development and delivery of cloud-based solutions
• Collaborate with teams specializing in perception, planning, control, mapping and vehicle testing to develop solutions that support product delivery
• Support the implementation of emerging cloud-based capabilities that can extend our technology stack and improve our ability to build, deploy and test safety-critical software for self-driving vehicles
• Participate in the team’s on-call rotation to support our deployed systems during business hours

Requirements:

• Bachelor's degree in Computer Science, Software Engineering, or related field with 6+ years of professional experience
• Master's degree in Computer Science, Software Engineering, or related field with 3+ years of professional experience
• Strong proficiency writing code using Python
• Practical experience with Docker and containerization
• A strong commitment to test-driven development patterns, continuous integration and delivery, and infrastructure as code
• Experience with Linux and general bash scripting
• Experience deploying, troubleshooting, monitoring and maintaining Linux systems
• Strong organizational, time management, and communication skills working with a team orientation and collaborative style
• Experience developing cloud-based serverless solutions
• Experience with pandas, numpy and other Python-based data analysis libraries and tooling
• Deep knowledge of AWS serverless architectures (Lambda, Batch, ECS Fargate, Glue, Athena)
• Experience with data storage and acquisition patterns for robotics and advanced driver assistance systems
• Knowledge of different database architectures, including but not limited to relational and NoSQL databases, vector stores, data warehousing and clustered, distributed data stores
• Experience with the Databricks platform, particularly for serving data, visualizations and jobs
• Experience with scaling data for ML and AI workloads using Ray