Key skills
Machine LearningData EngineeringMLOpsApache SparkSpark Structured StreamingRay DataFeature StoreTraining ML ModelsFine-tuning LLMsOnline ML InferenceOffline ML InferenceKubernetesAWS EKSIstioDatadogTerraformCloudFlareHelmJavaSpring BootGradleDynamoDBKinesisAirFlowPostgresPlanetscaleRedisReactTypeScriptGraphQLStorybookRadix UIViteesbuildPythonMetaflowHuggingFacePyTorchTensorFlowPandasAIMLSparkKafka
About this role
Attentive is the AI marketing platform for 1:1 personalization redefining the way brands and people connect. They are seeking a Senior Software Engineer for their Machine Learning Platform team, responsible for building and operating the ML data, tooling, serving, and inference layers of the ML platform to enhance their AI product suite.
Responsibilities:
- Unlock offline & real-time access to trillions of data points for our ML and Data Science teams
- Manage, expand, and optimize our feature store that enables feature engineering, multi-TB scale training jobs, and offline / real-time inferencing
- Support PB scale data operations on the feature store using Apache Spark, Spark Structured Streaming, Kafka, and Ray
- Partner with other teams and business stakeholders to deliver ML and AI initiatives
Requirements:
- You have been working in the areas of Data Engineering / MLOps for 5+ years, and have built and matured the pipelines of a PB-scale feature store
- You have deep Apache Spark, Spark Streaming, and Ray Data experience and built data pipelines for ML use cases using these tools
- You understand the correlation between data cardinality, query plans, configuration settings, and hardware and the impact of each on data pipeline performance
- You know/have created infrastructure for Training ML models/fine-tuning LLMs
- You understand the key differences between online and offline ML inferences and can voice the critical elements to be successful with each to meet business needs