Senior Data Engineer

Bright Vision Technologies is a technology consulting and software development company delivering cloud, AI, data, and enterprise solutions across the United States. The Senior Data Engineer will architect, develop, and maintain enterprise data platforms, scalable pipelines, and distributed processing systems supporting analytics, business intelligence, and machine learning. The role covers the full data engineering lifecycle and involves collaboration with cross-functional teams in an Agile environment.

Responsibilities:

• Architect, design, develop, and maintain enterprise-grade data platforms, scalable data pipelines, and distributed data processing systems that support analytics, business intelligence, and machine learning initiatives across multiple business domains
• Responsible for the end-to-end data engineering lifecycle, from translating business and analytical requirements into robust data architectures, to developing reliable ETL/ELT pipelines, to deploying cloud-native data solutions and supporting them throughout their operational lifespan
• Work closely with data scientists, business analysts, software engineers, solution architects, DevOps engineers, and stakeholders in an Agile environment to deliver high-quality, reliable, and governed data platforms that directly support strategic business outcomes

Requirements:

• Bachelor's degree in Computer Science, Information Technology, Data Engineering, Software Engineering, Mathematics, or a closely related technical discipline
• Five or more years of professional experience designing, developing, and supporting production-grade enterprise data engineering solutions, ETL pipelines, and cloud-based data platforms
• Strong, demonstrable understanding of data structures, database design, distributed computing, data modeling, ETL/ELT methodologies, data warehousing concepts, and large-scale data architecture principles
• Advanced working knowledge of Python, SQL, Spark, Scala, Java, and enterprise data engineering frameworks used to build scalable, high-performance data processing solutions
• Hands-on, production-level experience designing and operating batch processing, streaming data pipelines, data lakes, and cloud-native data platforms using technologies such as Databricks, Snowflake, Apache Spark, Kafka, Airflow, or equivalent solutions
• Proven experience working with relational and NoSQL databases including PostgreSQL, SQL Server, Oracle, MySQL, MongoDB, Cassandra, or equivalent database technologies, including schema design, query optimization, indexing strategies, and performance tuning
• Strong SQL skills and meaningful experience designing dimensional models, star schemas, snowflake schemas, data marts, partitioning strategies, indexing, and enterprise-scale data warehouse solutions
• Solid experience with Git-based version control, CI/CD pipelines, DevOps practices, release management, infrastructure automation, and Agile software development methodologies supporting enterprise data engineering initiatives
• Hands-on experience deploying enterprise data platforms and analytics solutions on AWS, Azure, or Google Cloud Platform, including managed storage, compute, networking, security, identity management, and data integration services
• Strong troubleshooting, analytical thinking, debugging, root-cause analysis, communication, and documentation skills, with the ability to investigate complex data processing issues methodically and implement scalable, maintainable engineering solutions
• Experience designing and implementing event-driven architectures, real-time data streaming platforms, Apache Kafka, Apache Flink, Apache NiFi, RabbitMQ, or equivalent enterprise messaging and streaming technologies
• Familiarity with containerization, orchestration, Infrastructure as Code, and cloud-native deployment practices using Docker, Kubernetes, Terraform, Helm, or equivalent enterprise automation technologies
• Exposure to distributed systems concepts including eventual consistency, fault tolerance, distributed transactions, data replication, partitioning strategies, CAP theorem, high availability, and large-scale data processing architectures
• Experience implementing data governance frameworks, master data management (MDM), data lineage, metadata management, data quality automation, security compliance, and DataOps best practices within enterprise cloud and Agile development environments