Senior Data Engineer

Job Summary We are seeking a Senior Data Engineer to design and build scalable data pipelines that support advanced analytics and machine learning solutions. This role involves ingesting, cleansing, and transforming data to make it analysis-ready, as well as operationalizing machine learning models for production use. The ideal candidate will have deep expertise in big data technologies, distributed systems, and cloud platforms, along with strong experience in building robust, high-performance data engineering solutions. Key Responsibilities Design and develop data pipelines to ingest, cleanse, and standardize data for analytics and machine learning use cases Collaborate with Data Scientists and Data Modelers to implement data processing and model deployment workflows Prepare and process data for production-ready machine learning applications Deploy and operationalize models through microservices, dashboards, reporting tools, or other delivery mechanisms Build and maintain scalable data infrastructure using big data and cloud technologies Implement DevOps practices, including source control, continuous integration, and automated deployments Develop and manage containerized applications using tools such as Docker and Kubernetes Work with distributed data systems and big data frameworks to ensure high performance and reliability Develop and integrate REST APIs and manage API authentication mechanisms Work with both relational (RDBMS) and NoSQL databases Ensure data quality, performance optimization, and system scalability Required Qualifications Bachelors degree in Computer Science, Engineering, or a related field, or equivalent practical experience 10+ years of experience in data engineering, with a focus on big data or cloud-based data pipelines Strong experience with big data technologies and distributed systems Hands-on experience with the Hadoop ecosystem, including HDFS, MapReduce, Hive, Pig, Impala, Spark, Kafka, Kudu, and Solr Experience with DevOps practices, including source control, CI/CD, and deployment pipelines Experience with containerization technologies such as Docker and Kubernetes Strong understanding of data structures, algorithms, and distributed computing Experience with REST APIs and service-based architectures Knowledge of relational and NoSQL database technologies Strong problem-solving and analytical skills Preferred Qualifications Experience with open-source tools such as Druid, Elasticsearch, and Logstash Experience with microservices architecture and best practices Experience working in large-scale, enterprise data environments Familiarity with performance tuning and optimization of data systems Education: Bachelors Degree