Principal Software Engineer, AI Platform Engineering

Saviynt is a leader in identity security, delivering an AI-powered platform that governs and secures access to applications, data, and business processes for global enterprises and government institutions. The Principal Software Engineer will set the architectural direction for training data governance across the AI Platform and define standards for ML engineers and scientists, ensuring data is tenant-isolated, PII-free, and traceable.

Responsibilities:

• AI Data Lake on GCS: bucket layout, raw → silver → gold tier separation, CMEK encryption, lifecycle rules
• Batch pipelines: Spark on Dataproc for TB-scale feature backfills, Iceberg compaction, and daily S3→GCS incremental sync
• Streaming pipelines: Apache Beam on Dataflow for sub-5-min CDC ingestion with exactly-once semantics and PII assertion gates
• Schema registry: Avro / Protobuf schema versioning, compatibility modes, and migration playbooks for safe schema evolution
• Orchestration: Flyte as primary DAG layer — task authoring standards, domain isolation, retry policies, DataCatalog memoization; evaluate Kubeflow Pipelines where relevant
• Multi-tenancy: strict per-tenant GCS prefix isolation, quota policies, and cross-tenant contamination validation
• Data Anonymizer and Data Labeler microservices: strip PII and attach ML labels before signals leave each customer environment
• Feature store: Feast offline (GCS Parquet) and online (Redis) with point-in-time correctness and < 0.1% consistency SLA
• Vector database: operate Pgvector (Cloud SQL) for POC and Qdrant on GKE for production-scale embedding storage; design index strategies (IVFFlat, HNSW) and manage ANN query latency SLAs
• RAG data pipeline: build embedding generation pipelines that chunk, encode, and upsert document embeddings into the vector store; own the data refresh cadence and staleness SLAs for retrieval context
• Service APIs: expose data platform services (feature serving, embedding upsert, schema validation) over HTTPS with mTLS and gRPC where low-latency streaming is required
• Synthetic data pipelines for dev/staging where real customer data is not permitted
• Data quality gates: Great Expectations / dbt checks as Flyte tasks, blocking on schema and PII-absence failures