Data Engineer, AI & Distributed Systems

Zignal Labs is a company that provides real-time intelligence technology to help organizations protect their people and assets. They are seeking a Data Engineer to build and operate data pipelines that process unstructured data and support their AI systems, ensuring reliable and efficient data flow for mission-critical decision making.

Responsibilities:

• Build and maintain pipelines. Develop and operate batch and streaming pipelines that ingest and enrich high-volume unstructured data. Own components end to end, from implementation through production monitoring
• Support our AI systems. Build and extend the data pathways that feed downstream NLP, LLM, and retrieval services — including data preparation, embedding generation, and indexing workflows
• Work with search and storage layers. Integrate with and tune our search and vector stores to support semantic search, clustering, and real-time retrieval
• Build services and APIs. Implement and improve the microservices and APIs that deliver analytics to enterprise customers
• Operate what you build. Write clean, tested, maintainable code. Participate in code review, CI/CD, and infrastructure-as-code practices. Debug production issues and improve reliability over time
• Collaborate across teams. Work with Data Science, ML, Product, and Security to take ideas from prototype into production

Requirements:

• 3+ years building and operating data pipelines in production
• Strong programming skills in a JVM language — Scala, Java, or Kotlin. Our core pipeline code is Scala. If you're strong in Java or Kotlin and want to learn Scala, we'll support that; we care more about your fundamentals than your current syntax
• Working proficiency in Python for data and scripting work
• Hands-on experience with a distributed processing framework, most likely Apache Spark
• Hands-on experience with a streaming platform, most likely Kafka — including a real understanding of consumer groups, offsets, partitioning, and what happens when things fall behind
• Practical AWS experience and comfort with Docker. You should be able to work in a Kubernetes environment; you don't need to administer one
• Experience with a workflow orchestrator such as Airflow, Prefect, or Dagster
• Solid SQL and experience with at least one NoSQL or caching layer (Redis, MongoDB, DynamoDB, or similar)
• Sound CS fundamentals — data structures, algorithms, and the judgment to reason about performance and correctness in a distributed setting
• Strong written communication and the ability to work asynchronously across U.S. time zones. We're fully remote; writing clearly is part of the job
• Databricks or Delta Lake specifically
• Flink, or other stream-processing frameworks beyond Kafka
• Vector databases (Pinecone, Qdrant, Milvus, pgvector) and hands-on RAG or embedding pipeline work
• Elasticsearch or OpenSearch
• Experience parsing messy, unstructured, or multilingual text at scale
• Deep database performance tuning, or experience with distributed consensus systems
• Bachelor's degree in Computer Science, Engineering, or a related field