Bright Vision Technologies is a forward-thinking software development company dedicated to building innovative solutions that help businesses automate and optimize their operations. They are seeking a highly skilled Senior Data Engineer to design, build, and optimize data pipelines and architectures, ensuring data quality and reliability while collaborating with various stakeholders.
Responsibilities:
- Design, develop, and maintain scalable ETL/ELT pipelines for structured and unstructured data
- Architect and optimize data warehouses, data lakes, and lakehouse solutions
- Collaborate with data scientists, analysts, and business stakeholders to ensure data availability, quality, and reliability
- Implement data governance, security, and compliance standards across platforms
- Monitor and improve data pipeline performance, scalability, and fault toler
- Work with cloud platforms (AWS, Azure, GCP) to deploy and manage data solutions
- Integrate data from diverse sources including APIs, streaming platforms, and enterprise systems
Requirements:
- Bachelor's or Master's degree in Computer Science, Information Technology, or related field
- 7+ years of experience in data engineering, big data, or related roles
- Strong proficiency in: SQL, Python, Scala, or Java
- Big Data frameworks: Spark, Hadoop, Kafka
- Cloud services: AWS (Glue, Redshift, EMR), Azure (Data Factory, Synapse), GCP (BigQuery, Dataflow)
- Data modeling and warehousing: Snowflake, Redshift, Synapse, BigQuery
- Experience with workflow orchestration tools (Airflow, Prefect, Dagster)
- Solid understanding of data governance, lineage, and security practices
- Hands-on experience with CI/CD pipelines and DevOps practices for data engineering
- Exposure to machine learning pipelines and MLOps
- Experience with containerization (Docker, Kubernetes)
- Knowledge of real-time data streaming and event-driven architectures
- Relevant certifications (AWS Data Analytics, Azure Data Engineer Associate, GCP)