Develop robust, scalable data architecture that supports data teams (Analytics, BI, Data Science, Machine Learning and other verticals) and the business, democratizing access to data in a secure, simple, structured and high-quality manner;
Design, build and integrate monitoring and oversight mechanisms for the data ecosystem;
Design/build development standards and best practices, and document the environment and deliverables;
Document, monitor and ensure data quality;
Design and ensure the application of best practices for pipeline development, focusing on performance and scalability;
Ensure and maintain a view capable of relating/integrating areas (Data Products and MLOps) and data solutions;
Ensure knowledge is shared across the entire team;
Identify, manage and address risks related to activities under your responsibility and your team’s;
Ensure and implement the designed data architecture;
Ensure correct understanding of business needs and strategic vision;
Ensure that all mapped requirements to be developed are delivered with quality, scalability and on schedule;
Requirements
Bachelor’s degree in Engineering, Computer Science or related fields;
Experience with Azure and/or AWS;
Experience with Databricks;
Experience with Delta Lake / Lakehouse architectures;
Experience with Cosmos DB, Azure Synapse and Analysis Services;
Proficiency in Python, SQL and PySpark;
Experience with distributed data processing;
Experience with continuous integration and continuous delivery (CI/CD) and/or Azure DevOps;