Machine Learning Engineer / Data Scientist

SullivanCotter is a company that enables health care organizations to build thriving clinician teams through innovative technology products and analytics. They are seeking a highly skilled Machine Learning Engineer to develop and deploy cutting-edge machine learning models and data analytics solutions, collaborating with cross-functional teams to drive data-informed decision-making.

Responsibilities:

• Design, develop, and deploy ML solutions ranging from traditional ML applications (classification, clustering, recommendations) to LLM-based systems, including document parsing, data extraction, RAG pipelines, and LLM agents
• Write clean, maintainable, production-quality Python code that integrates smoothly with existing engineering and deployment infrastructure
• Work with large datasets to clean, preprocess, and analyze data, ensuring data quality and integrity
• Implement and optimize algorithms using best practices in machine learning, deep learning, and statistical analysis
• Collaborate with business stakeholders to understand requirements and deliver data-driven solutions that provide actionable insights
• Develop and maintain scalable pipelines and infrastructure for data processing and model training, versioning, deployment, and monitoring
• Evaluate the performance of machine learning models, including LLM-specific evaluation approaches, and tune models for optimal performance
• Communicate findings, insights, and model performance to both technical and non-technical audiences
• Continuously stay updated on the latest trends, technologies, and best practices

Requirements:

• Bachelor's or Master's degree in Computer Science, Data Science, Statistics, Mathematics, or a related field or related experience
• Bachelor with 5+ years of relevant experience
• Master or higher with 3+ years of relevant experience
• Fluent in Python (3+ years of coding experience)
• Strong software development practices in Python, including writing maintainable, testable, production-ready code
• Solid understanding of LLM architectures and Generative AI
• Hands-on experience building and evaluating RAG pipelines
• Experience with LLM orchestration frameworks (LangChain, LlamaIndex, or similar)
• Proficiency in machine learning libraries such as Scikit-learn and PyTorch; and fundamental libraries such as NumPy and Pandas
• Familiarity with cloud platforms (e.g., AWS, GCP, Azure) and containerization tools (e.g., Docker)
• Strong understanding of model evaluation metrics across traditional ML (e.g., accuracy, precision, recall, F1) and LLM-based systems (e.g., faithfulness, answer relevancy, hallucination detection), including approaches for evaluating non-deterministic outputs
• Experience with model management tools such as MLFlow and the model development life cycle
• Experience with version control tools such as Git
• Proficiency in adapting SDLC best practices for code development and testing
• Excellent problem-solving skills, analytical thinking, and the ability to work in a fast-paced environment
• Strong communication skills and the ability to explain complex technical concepts to non-technical stakeholders
• Familiarity with optimizing, deploying and scaling automated training pipelines of transformer-based models
• Familiarity with distributed training techniques and GPU-accelerated computing
• Familiarity with classical NLP approaches
• Experience implementing CI/CD pipelines for ML models for automating training, validation, monitoring, and scalable deployment
• Experience with integrating and deploying AWS AI/ML services
• Experience with Databricks
• Experience in Health Care data