About this role

Develop PySpark Notebooks within Microsoft Fabric.
Build, monitor, and support data pipelines and Dataflow Gen2 processes.
Write and optimize SQL queries to support data transformations and analytics.
Manage environment variables and configuration settings across development environments.
Implement and maintain data quality rules, monitoring, and exception handling across bronze, silver, and gold data layers.
Maintain documentation of data pipelines, from ingest to gold layer.
Prepare semantic models for reporting.
Implement role-based access controls and data security models across Lakehouses and semantic models.
Collaborate with team members using Azure DevOps and/or GitHub for version control and deployment.
Validate data quality/accuracy and assist in troubleshooting data issues.
Follow established data engineering standards and best practices.
Perform other duties as assigned.

1–3 years of experience in data engineering, data analytics, business intelligence, or other data-related roles.
Exposure to Microsoft Fabric, Azure, and Lakehouses or similar cloud data platforms.
Experience with PySpark or Apache Spark concepts.
Understanding of CI/CD principles for data or software projects.
Experience in SaaS, GovTech, or technology-enabled services organizations.

BI Engineer I

Key skills