Senior Manager – Data Engineering

Tredence Inc. is seeking a Senior Manager – Data Engineering to lead the architecture and implementation of scalable cloud-based data platforms. The role focuses on data modeling, ETL/ELT pipelines, big data processing, cloud analytics integration, performance optimization, CI/CD, migration support, and data governance and security.

Responsibilities:

• Design and architect scalable data platforms in the cloud, ensuring they meet both technical and business requirements
• Lead the technical vision for cloud-based data solutions, aligning architecture and implementation with strategic goals
• Implement and optimize star schema models within cloud data warehouses, focusing on performance and scalability
• Regularly review data workloads to identify opportunities for operational and performance improvements
• Manage capacity in cloud data warehouses by proactively adjusting resources based on usage patterns and demand
• Monitor system performance and ensure optimal operation through tuning and resource management
• Build high-performance data models by applying effective table distribution and indexing strategies
• Implement and manage cloud-based data lakes, leveraging hierarchical namespaces and POSIX-compliant access control mechanisms to ensure secure and organized data storage
• Design, develop, and optimize ETL/ELT pipelines for efficient data movement and transformation
• Implement big data processing solutions to handle large-scale datasets, and automate workflows to support real-time data streaming and analytics
• Integrate cloud-based analytics platforms with other services such as data lakes, orchestration tools, and data warehouses to create seamless data ecosystems
• Manage and maintain cloud compute clusters, optimizing resource utilization for both performance and cost efficiency
• Develop and maintain CI/CD pipelines to support continuous integration and deployment of data processes
• Execute change tasks such as table partitioning using appropriate strategies to enhance query performance and manage data lifecycle effectively
• Identify and resolve performance bottlenecks across the cloud data environment, ensuring systems run efficiently and reliably
• Provide guidance and documentation to developers and end-users on best practices for cloud services, including maintenance and housekeeping
• Address incidents and issues related to data migration from on-premises systems to the cloud, ensuring smooth transitions and minimal disruption
• Uphold data quality, governance, and security by implementing industry best practices and compliance standards

Requirements:

• Master's degree (or foreign equivalent) in Information Systems, or a related field, PLUS two (2) years of experience in the job offered or a related position
• Experience must include demonstrable knowledge of: SQL; Python; ETL/ELT data pipelines; partitioning tables; Star & Snowflake data models; CI/CD pipelines; AWS S3; Databricks; JIRA, and; Confluence
• Required knowledge may be gained prior to or concurrently with Master's degree
• Travel to unanticipated client locations throughout the U.S., approximately 30%, as required
• May telecommute from any location within the U.S