You will plan, develop and support the data engineering solutions that prepare, integrate and make data accessible for analytical and operational use across ASD
You will work across multiple data sources and formats, build reusable NiFi components, and shape the data flow and integration strategies that the broader analytical capability depends on
Plan and drive the development of data engineering solutions, balancing functional and non functional requirements
Build scalable and reusable NiFi templates and processors for data ingestion and transformation
Optimise performance and ensure high availability of data pipelines
Integrate data from multiple sources and formats into centralised systems
Perform data modelling to support efficient data storage, retrieval and analytics
Provide expert advice on data integration and uplift the capability of others in applying good integration practice
Monitor application of data standards, architectures and security to ensure compliance and scalability
Develop standards and guidelines for data protection, disaster recovery and storage management
Contribute to organisational policies and guidelines for data engineering
Collaborate with cross functional teams to define data flow and integration strategies
Lead the selection and development of data engineering methods, tools and techniques
Develop and promote continuous integration, deployment and monitoring practices

Expert hands on experience with Apache NiFi, including building scalable, reusable templates and processors for ingestion and transformation at enterprise scale
Data engineering capability at SFIA Level 6, leading data engineering activities for strategic, large and complex programmes and setting technical direction
Strong data modelling and design at SFIA Level 5, including managing enterprise data requirements and setting modelling standards
Storage management at SFIA Level 5, including data protection, disaster recovery and managing storage and backup systems to deliver agreed service levels
You must hold a current PV security clearance and be an Australian citizen to be considered for this role

Senior Data Engineer – NiFi

Key skills