Key skills
PythonJavaKotlinScalaShellSparkAirflowHadoopCassandraAWSJenkinsECSCloudFormationEC2S3API GatewayGitGitHubGitLabBitbucketCI/CD
About this role
Recurring Decimal is seeking a Data Engineer to join their team. The role involves building and maintaining Big Data applications on AWS, developing REST APIs, and optimizing performance for various data processing tasks.
Responsibilities:
- 10+ years in analysis, coding, implementation, and testing of Big Data applications (API & batch) on AWS
- Strong hands-on experience with Java/Scala/Kotlin, Python, Spark, Hadoop, HDFS, MapReduce, Cassandra
- Build and maintain REST APIs and SOAP services using Scala, PySpark, and Java
- Develop AWS-based batch and microservices using EMR, EC2, ECS, S3, Airflow, Step Functions, API Gateway
- Performance tuning for Spark/Hadoop/EMR, Java, and Python batch jobs
- Solid Linux experience, including shell scripting and Python
Requirements:
- 10+ years in analysis, coding, implementation, and testing of Big Data applications (API & batch) on AWS
- Strong hands-on experience with Java/Scala/Kotlin, Python, Spark, Hadoop, HDFS, MapReduce, Cassandra
- Build and maintain REST APIs and SOAP services using Scala, PySpark, and Java
- Develop AWS-based batch and microservices using EMR, EC2, ECS, S3, Airflow, Step Functions, API Gateway
- Performance tuning for Spark/Hadoop/EMR, Java, and Python batch jobs
- Solid Linux experience, including shell scripting and Python
- DevOps tools: Git, GitHub, Bitbucket, GitLab, IntelliJ, PyCharm
- CI/CD: Maven, Gradle, Jenkins, Artifactory, AWS CodeCommit, CloudFormation