Key skills
Data EngineeringAWS GluePySparkPythonSQLAWS S3AWS LambdaAWS Step FunctionsAWS AthenaAWS RDS PostgreSQLAWS AuroraAWS EventBridgeAWS SNSAWS SQSAWS SESAWS CloudWatchAWS SCTAWS DMSUNIX Shell scriptingUNIX/Linux environmentsData modelingPerformance tuningData migrationETL pipelinesShellETLData WarehousingData LakeAnalyticsBISparkApache SparkAWSServerlessLambdaS3RDSCloudWatchSNSSQSGlueAthenaPostgreSQLCI/CDCommunicationCollaboration
About this role
Cognizant is seeking a highly experienced Technical Lead – Data Engineering with expertise in delivering enterprise-scale data solutions on AWS. This role involves designing, building, and modernizing data platforms, developing complex data engineering solutions, and contributing to cloud transformation initiatives.
Responsibilities:
- Design, build, and optimize scalable data engineering solutions using AWS cloud services
- Develop and maintain robust ETL pipelines using AWS Glue (PySpark/Python) to process large-scale data efficiently
- Implement and enhance data integration frameworks leveraging Glue Data Catalog, Crawlers, and S3/JDBC sources
- Develop high-performance data processing logic using PySpark, Python, and SQL
- Contribute to cloud data migration initiatives from on-premises systems to AWS (RDS PostgreSQL/Aurora) using AWS SCT, DMS, or custom frameworks
- Orchestrate and manage data workflows using AWS Step Functions, Glue Workflows, and event-driven services
- Collaborate with cross-functional teams to understand data requirements and deliver scalable data solutions
- Monitor, troubleshoot, and optimize data pipelines using CloudWatch and other AWS monitoring tools
- Ensure data quality, performance, and reliability across all data engineering processes
- Follow best practices in coding, testing, and deployment to maintain high-quality deliverables
Requirements:
- 10+ years of experience in IT with a strong focus on Data Engineering and Analytics
- Proven hands-on experience designing and developing scalable data pipelines using Apache Spark (PySpark)
- Strong expertise in AWS services, including Glue, S3, Lambda, Step Functions, Athena, RDS (PostgreSQL/Aurora), EventBridge, SNS, SQS, SES, and CloudWatch
- Extensive experience building and optimizing ETL pipelines using AWS Glue (PySpark/Python)
- Experience in data migration from on-premises systems to AWS using tools such as AWS SCT and DMS or custom frameworks
- Strong programming skills in Python, SQL, and UNIX Shell scripting
- Experience working with Glue Data Catalog, Glue Crawlers, and handling structured and unstructured data sources
- Solid understanding of data modeling, performance tuning, and optimization techniques for large-scale datasets
- Hands-on experience with UNIX/Linux environments
- Ability to work independently in a fast-paced environment and contribute to complex technical solutions
- Bachelor's degree in Computer Science, Engineering, or a related field (or equivalent practical experience)
- Experience with serverless and event-driven architectures on AWS
- Strong understanding of data lake and data warehousing concepts
- Hands-on experience with performance tuning and cost optimization in AWS data pipelines
- Familiarity with reporting and BI tools such as Cognos
- Experience working with large-scale, high-volume data processing systems
- Exposure to CI/CD pipelines and DevOps practices for data engineering workflows
- Strong problem-solving skills with the ability to troubleshoot complex data and system issues
- Excellent collaboration and communication skills in a cross-functional environment