Lead Data Engineer

Job Summary:

We are seeking a highly skilled Lead Data Engineer to join our dynamic team. The ideal candidate will possess strong hands-on experience in advanced Oracle RDMS, SQL, PL/SQL, and various data engineering tools and technologies. This role requires a deep understanding of data architecture, data modeling, and the ability to work collaboratively in a fast-paced environment. The Lead Data Engineer will be responsible for designing, implementing, and maintaining data pipelines and ensuring data integrity and availability for analytics and reporting.

Responsibilities:

• Design, develop, and optimize complex SQL queries and PL/SQL scripts for data extraction, transformation, and loading (ETL).
• Create and manage Oracle database objects including tables, views, indexes, and partitioned objects.
• Implement and manage file transfers using NDM (Connect:Direct) to ensure secure and efficient data movement.
• Utilize Autosys for job scheduling and monitoring to ensure timely execution of data processes.
• Develop and maintain data processing workflows using Python and PySpark for large-scale data processing.
• Leverage GitHub as a code repository for version control and collaboration on data engineering projects.
• Utilize GitHub CoPilot to enhance development efficiency and streamline coding processes.
• Collaborate with cross-functional teams to gather requirements and deliver data solutions that meet business needs.
• Ensure data quality and integrity through rigorous testing and validation processes.
• Stay updated with industry trends and best practices in data engineering and contribute to continuous improvement initiatives.

Mandatory Skills:

• Advanced Oracle RDMS expertise, including object creation (tables, views, indexes, partitioned objects).
• Proficient in SQL and PL/SQL for data manipulation and analysis.
• Strong experience with K shell scripting, including sed, awk, grep, and invoking SQL Plus shell.
• Hands-on experience with NDM (Connect:Direct) for file transfers.
• Familiarity with Autosys for job scheduling and monitoring.
• Proficient in Python and PySpark for data processing and analytics.
• Experience using GitHub as a code repository.
• Comfortable using GitHub CoPilot for AI development efficiencies.

Preferred Skills:

• Experience with data warehousing concepts and technologies.
• Knowledge of cloud-based data solutions (e.g., AWS, Azure, Google Cloud).
• Familiarity with data visualization tools (e.g., Tableau, Power BI).
• Understanding of data governance and compliance standards.