Data Engineer

Confidential is seeking a Data Engineer to develop and support scalable data platforms, pipelines, and automation solutions. The role is responsible for building and optimizing production-grade data pipelines, integrating multiple data sources, troubleshooting production issues, and improving data quality, reliability, and engineering productivity.

Responsibilities:

• Design, develop, test, and maintain scalable data pipelines and ETL/ELT workflows
• Build data processing solutions using Python, SQL, and related technologies
• Integrate data from multiple databases, APIs, files, and engineering systems
• Develop reusable and maintainable data-processing frameworks and automation tools
• Optimize data pipelines for performance, scalability, and reliability
• Develop complex SQL queries, stored procedures, and data transformations
• Work with relational databases and large-scale data systems
• Perform data validation, cleansing, transformation, and quality checks
• Troubleshoot database, query, and data-processing issues
• Support data modeling and schema design as needed
• Monitor production data pipelines, scheduled jobs, and data workflows
• Troubleshoot failures across applications, databases, pipelines, and infrastructure
• Improve logging, monitoring, alerting, and operational supportability
• Identify root causes and implement long-term solutions to recurring issues
• Maintain documentation and support knowledge-transfer activities
• Develop automation to reduce manual data-processing and operational tasks
• Use AI-assisted development tools to accelerate coding, testing, troubleshooting, and documentation
• Maintain ownership of data accuracy, security, reliability, and production readiness
• Partner with software engineers, data scientists, and engineering stakeholders to understand data requirements
• Translate business and engineering requirements into practical data solutions
• Participate in design reviews, code reviews, troubleshooting, and production-support activities
• Communicate technical issues, project status, risks, and recommendations clearly

Requirements:

• BS/MS in Computer Science, Data Engineering, Software Engineering, Electrical Engineering, or related field, or equivalent experience
• Strong Python development skills.
• Strong SQL skills and experience with relational databases.
• Experience building and maintaining ETL/ELT or data-processing pipelines
• Experience with production-quality software and data systems
• Experience working with databases, APIs, and multiple data sources
• Experience with Git or similar version-control systems
• Strong debugging and problem-solving skills
• Experience working in Linux/Unix environments
• Strong written and verbal communication skills
• Experience with Oracle, PostgreSQL, MySQL, or other enterprise databases
• Experience with AWS, Azure, or other cloud platforms
• Experience with Snowflake, Databricks, Spark, Kafka, Airflow, or similar data technologies
• Experience developing REST API integrations
• Experience with Docker and CI/CD tools such as Jenkins or GitHub Actions
• Experience with data quality, observability, monitoring, and logging
• Experience with distributed computing or batch-processing environments
• Experience in semiconductor, hardware, manufacturing, or engineering environments
• Experience using AI/LLM-assisted development tools