Key skills
Data EngineeringPythonAWS servicesSQL programmingCI/CD DevOpsData modelingAgile/SCRUMInter-personal skillsCommunication skillsSQLETLData WarehousingKafkaData MiningUnit TestingAWSJenkinsEKSLambdaS3RDSGluePostgresOracleSQL ServerGitHubBitbucketAgileScrumCI/CDCommunication
About this role
Innovatech Staffing is seeking a Data Engineer to be a critical member of their data team. The role involves designing data integration solutions that deliver business value, focusing on data processing, manipulation, and ETL using Python/Pyspark and AWS tools.
Responsibilities:
- Provide scoping, estimating, planning, design, development, and support services to a project
- Identify and develop the Technical detail design document
- Work with developers and business areas to design, configure, deploy and maintain custom ETL Infrastructure to support project initiatives
- Design and develop data/batch processing, data manipulation, data mining, and data extraction/transformation/loading (ETL Pipelines) into large data domains
- Document and present solution alternatives to clients, which support business processes and business objectives
- Work with business analysts to understand and prioritize user requirements
- Design, development, test, and implement application code
- Follow proper software development lifecycle processes and standards
- Quality Analysis of the products, responsible for the Defect tracking and Classification
- Track progress and intervene as needed to eliminate barriers and ensure delivery
- Resolve or escalate problems and manage risk for both development and production support
- Maintain deep knowledge and awareness of technical & industry best practices and trends, especially in technology & methodologies
Requirements:
- At least 10+ years of Developer experience specifically focused on Data Engineering
- Strong Hands-on experience in Data Engineering development using Python and Pyspark as an ETL tool
- Hands-on experience in AWS services like Glue, RDS, S3, Step functions, Event Bridge, Lambda, MSK (Kafka), EKS etc
- Hands-on experience in Databases like Postgres, SQL Server, Oracle, Sybase
- Hands-on experience with SQL database programming, SQL performance tuning, relational model analysis, queries, stored procedures, views, functions and triggers
- Strong technical experience in Design (Mapping specifications, HLD, LLD), Development (Coding, Unit testing)
- Good knowledge in CI/CD DevOps process and tools like Bitbucket, GitHub, Jenkins
- Strong foundation and experience with data modeling, data warehousing, data mining, data analysis and data profiling
- Strong experience with Agile/SCRUM methodology
- Good communication and inter-personal skills