Senior Data engineer

Bright Vision Technologies is a forward-thinking software development company dedicated to building innovative solutions that help businesses automate and optimize their operations. They are seeking a highly skilled Senior Data Engineer to design, build, and optimize data pipelines and architectures, ensuring data quality and reliability while collaborating with various stakeholders.

Responsibilities:

• Design, develop, and maintain scalable ETL/ELT pipelines for structured and unstructured data
• Architect and optimize data warehouses, data lakes, and lakehouse solutions
• Collaborate with data scientists, analysts, and business stakeholders to ensure data availability, quality, and reliability
• Implement data governance, security, and compliance standards across platforms
• Monitor and improve data pipeline performance, scalability, and fault toler
• Work with cloud platforms (AWS, Azure, GCP) to deploy and manage data solutions
• Integrate data from diverse sources including APIs, streaming platforms, and enterprise systems

Requirements:

• Bachelor's or Master's degree in Computer Science, Information Technology, or related field
• 7+ years of experience in data engineering, big data, or related roles
• Strong proficiency in: SQL, Python, Scala, or Java
• Big Data frameworks: Spark, Hadoop, Kafka
• Cloud services: AWS (Glue, Redshift, EMR), Azure (Data Factory, Synapse), GCP (BigQuery, Dataflow)
• Data modeling and warehousing: Snowflake, Redshift, Synapse, BigQuery
• Experience with workflow orchestration tools (Airflow, Prefect, Dagster)
• Solid understanding of data governance, lineage, and security practices
• Hands-on experience with CI/CD pipelines and DevOps practices for data engineering
• Exposure to machine learning pipelines and MLOps
• Experience with containerization (Docker, Kubernetes)
• Knowledge of real-time data streaming and event-driven architectures
• Relevant certifications (AWS Data Analytics, Azure Data Engineer Associate, GCP)