Remote/Hybrid Full Time Only: AWS Data Engineer
United States of America
Full Time
15 hours ago
Visa Sponsor
Key skills
PythonPython packages developmentETL automationAirflow DAGData pipeline orchestrationData pipeline troubleshootingCSVJSONXMLS3AI IDE toolSQLSQL query optimizationSnowflakePython unit testingPython integration testingPython system testingDBT testsCode generation with Python and Jinja templatesGitHubCI/CD processAWS S3AWS ECSAWS IAMAWS security best practicesEncryption standardsSecure coding guidelinesDBT profiles access configurationAIETLAirflowdbtAWSECSIAMCloud StorageCI/CD
About this role
Visionary Innovative Technology Solutions LLC is seeking an AWS Data Engineer. The role involves developing and maintaining data pipelines, optimizing SQL queries, and ensuring data quality, while working with AWS services and tools.
Responsibilities:
- Must have below:
- 10+ Years Experience
- Great Communicator/Client Facing/ Attention to detail
- Individual Contributor and ability to work as a team
- 100% Hands on in the mentioned skills
Requirements:
- 10+ Years Experience
- Great Communicator/Client Facing/ Attention to detail
- Individual Contributor and ability to work as a team
- 100% Hands on in the mentioned skills
- Advanced Proficiency in Python concepts like Code Structures, Modules, Packages, Class, SubClass, Inheritance, Multi-Threading and Functional Programming
- Experience in developing reusable Python packages for internal or public usage
- Ability to write automating ETL processes and scheduling jobs like Airflow DAG
- Ability to track job pipeline runs to reprocess error records
- Ability to orchestration different pipelines to run in sequence or parallel
- Troubleshoot data pipeline errors and fix issues
- Export or Import data to/from various formats like CSV, JSON, XML etc preferably from S3 or other cloud storage
- Experience in using AI IDE tool
- Advanced SQL skills, including complex joins, CTE's and subqueries
- Experience in optimizing SQL queries for performance and optimization in data warehouse technologies preferably Snowflake
- Proficiency in Python unit, integration, and system test
- Proficiency in implementing DBT tests for data validation and quality checks
- Experience in generating code using configurations using python and jinja templates
- Experience in GitHub, including implementing CI/CD process from scratch
- In depth understanding of AWS S3 for data storage, ECS, IAM including best practices for organization and security
- Knowledge of AWS security best practices, including IAM roles, encryption standard, secure coding guidelines, DBT profiles access configurations and more