Key skills
Apache SparkDatabricksAWS Cloud servicesPythonData modelingData pipelinesJavaScalaETLELTData EngineeringData LakeAnalyticsRedshiftSparkAirflowAWSLambdaS3IAMGluePerformance OptimizationCommunicationCollaboration
About this role
Careforth is a pioneer in the caregiving space, supporting family caregivers across the United States. The Senior Data Engineer will contribute to the evolution of the enterprise DataHub/Delta Lake architecture and drive best practices in cloud-native, Spark-based data engineering.
Responsibilities:
- Architect and build robust, scalable, and secure data pipelines leveraging Databricks, Apache Spark, and AWS Cloud (EMR, Redshift, S3, Glue, Lambda)
- Participate and represent the data team in critical design discussions with technical leads across various product lines
- Collaborate closely with application developers, product managers, and business analysts to translate requirements into data models, ETL/ELT workflows, and analytics-ready datasets
- Conduct pull request reviews and enforce engineering excellence in code quality, testing, and performance optimization
- Troubleshoot and optimize production data workflows while ensuring observability, resilience, and cost-efficiency at scale
- Research, evaluate, and apply emerging tools and technologies to continuously modernize the data engineering ecosystem
- Act as a mentor to junior engineers, fostering a culture of collaboration, innovation, and continuous learning
Requirements:
- Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field
- 7-10 years of professional data engineering experience, including at least 3+ years with modern cloud-based data lake architectures
- Deep expertise in Apache Spark (PySpark, Scala, or Java) for large-scale distributed data processing
- Strong experience with Databricks for collaborative data engineering and advanced analytics
- Hands-on experience with AWS services including EMR, Redshift, S3, Glue, Lambda, IAM, and related cloud-native data tools
- Proficiency in Python (preferred) as well as Java or Scala
- Strong understanding of data modeling, data pipelines, and workflow orchestration (Airflow, Step Functions, or similar)
- NOGO experience
- Solid foundation in algorithms, data structures, and software engineering best practices
- Excellent communication skills and a proven ability to work cross-functionally with product and engineering teams