The Hadoop Architecture Lead is responsible for defining, driving, and governing the architecture of the enterprise Hadoop ecosystem.
This role ensures scalability, reliability, cost efficiency, and alignment with broader the Modern Data Platform strategy, observability tools, while working with tenants to enable advanced analytics, AI/ML, and data product use cases.
Sets and tracks quality and performance objectives by leveraging team and client feedback and facilitating performance and career development of individuals through performance reviews, coaching, and building individual development plans to improve competencies.
Builds and maintains teams through hiring and training, manages team resources and financials to ensure maturity objectives are met, and provides the necessary resources to enable team members to achieve them.
Manages relationships with business and technology leaders and vendors for technical products and creates an inclusive and healthy working environment to resolve organizational impediments and blockers.
Ensures efficacy of data and information solution delivery, including prioritizing technology debt, compliance, and security items and supports application performance in production, including application health, resiliency, security, enterprise data management standards, global records management standards, and audit exams and reviews.
Creates the technology strategy for a respective technical domain, aligning execution with product strategy by working with Product Managers, Product team members, and other stakeholders.
Ensures all relevant risk, financial, and compliance policies and standards are met.
Leads and creates followership in Communities of Practice in the organization.
Requirements
Extensive Engineering experience working w/ Hadoop, Kafka, Spark, Impala, Hive, Hbase etc.
Leadership/Management experience driving large‑scale data platform strategies across data lake, warehouse, and distributed computing systems.
Strong knowledge of Hadoop Architecture, HDFS, Hadoop Cluster and Hadoop Administrator's role
Intimate knowledge of fully integrated AD/Kerberos authentication
Experience setting up optimum cluster configurations
Debugging knowledge of YARN.
Hands-on with analyzing various Hadoop log files, compression, encoding, & file formats
Expert level knowledge of Cloudera Hadoop components such as HDFS, Sentry, HBase, Kafka, Impala, SOLR, Hue, Spark, Hive, YARN, Zookeeper and Postgres