Key skills
SQLNoSQLAIArtificial IntelligenceMachine LearningData EngineeringAnalyticsSparkAWSKubernetesDockerDynamoDBPostgreSQLElasticsearch
About this role
Rad AI is on a mission to transform healthcare with artificial intelligence, aiming to revolutionize radiology through innovative AI-driven solutions. They are seeking a Senior Data Engineer to design and implement data architectures, ensuring efficiency and scalability while collaborating with stakeholders to meet their data needs.
Responsibilities:
- Design and implement the data architecture, ensuring scalability, flexibility, and efficiency using pipeline authoring tools like Metaflow and large-scale data processing technologies like Spark
- Define and extend our internal standards for style, maintenance, and best practices for a high-scale data platform
- Collaborate with researchers and other stakeholders to understand their data needs including model training and production monitoring systems and develop solutions that meet those requirements
- Take ownership of key data engineering projects and work independently to design, develop, and maintain high-quality data solutions
- Ensure data quality, integrity, and security by implementing robust data validation, monitoring, and access controls
- Evaluate and recommend data technologies and tools to improve the efficiency and effectiveness of the data engineering process
- Continuously monitor, maintain, and improve the performance and stability of the data infrastructure
Requirements:
- 5+ years relevant experience in data engineering
- Expertise in designing and developing distributed data pipelines using big data technologies on large scale data sets
- Deep and hands-on experience designing, planning, productionizing, maintaining and documenting reliable and scalable data infrastructure and data products in complex environments
- Solid experience with big data processing and analytics on AWS, using services such as Amazon EMR and AWS Batch
- Experience in large scale data processing technologies such as Spark
- Expertise in orchestrating workflows using tools like Metaflow
- Experience with various database technologies including SQL, NoSQL databases (e.g., AWS DynamoDB, ElasticSearch, Postgresql)
- Hands-on experience with containerization technologies, such as Docker and Kubernetes
- Prior Software Engineering experience is a big plus
- Experience working at an early stage startup
- Experience in a HIPAA compliant environment
- Experience working on machine learning or healthcare related projects