Lead, mentor, and manage software engineers focused on data acquisition
Execute the technical roadmap and strategy for all data acquisition pipelines and systems
Provide technical guidance and oversight on the design, development, deployment, and maintenance of critical data ingestion frameworks, APIs, and services
Work closely with product management, data science, data governance, and other engineering teams to understand data needs and deliver solutions that meet business requirements
Drive continuous improvement in engineering processes, tools, and methodologies (e.g., CI/CD, automation, monitoring, alerting)
Ensure high availability and performance of all data acquisition systems, leading incident response and root cause analysis for any major outages
Manage budget, resources, and capacity planning for the data acquisition organization
Requirements
Bachelor's or Master's degree in Computer Science, Engineering, or a related technical field
10+ years of experience in software engineering
5+ years in a management role overseeing software engineering or data acquisition teams
Deep expertise in building and managing high-volume, real-time and batch data pipelines (e.g., Kafka, Kinesis, Pulsar)
Proficiency with cloud platforms (e.g., AWS, Azure, GCP)
Experienced in designing scalable data ingestion architectures (e.g., Kubernetes, EKS/AKS/GKE)
Strong knowledge of various data sources, integration patterns (APIs, web scraping, messaging queues), and ETL/ELT tools
Expertise in programming languages such as Java, Python, Scala, or Go
Solid understanding of database technologies (SQL, NoSQL, Data Warehouses like Snowflake, Redshift, etc.)
Proven ability to lead, motivate, and manage multiple distributed teams