Key skills
AWS Step FunctionsAWS GluePythonSQLGreat ExpectationsSodaAWS S3AWS LambdaAWS AthenaAWS IAMAWS CloudWatchParquetJSONAvroData EngineeringAWSLambdaS3IAMCloudWatchGlueAthenaVersion ControlCI/CDCommunication
About this role
gWorks is a company that builds software to help small local governments operate more efficiently. The Senior Data Engineer will be responsible for designing and building cloud-native data transformation pipelines, ensuring data quality and validation, and collaborating with various teams to support clients' needs.
Responsibilities:
- You'll design, build, and maintain scalable data transformation pipelines using AWS Step Functions and AWS Glue, orchestrating complex workflows that ingest, clean, and transform data from a variety of sources
- A core part of your work will involve implementing rigorous data validation frameworks at every stage of the pipeline, including schema enforcement, anomaly detection, completeness checks, and reconciliation against source systems
- You'll own the quality of the data you produce, defining SLAs, monitoring pipeline health, and proactively surfacing issues before they reach downstream users
- Beyond pipeline development, you'll collaborate with software engineers and product owners to understand their requirements, design appropriate data models, and ensure the outputs of your pipelines support both analytical and operational use cases
- You'll also contribute to platform-wide decisions around tooling, architecture, and best practices, and you'll mentor other team members on data engineering and validation patterns
Requirements:
- 5+ years of professional experience building production data transformation pipelines
- Strong hands-on experience with AWS Step Functions for workflow orchestration
- Strong hands-on experience with AWS Glue (Glue Jobs, Glue Catalog, Glue Crawlers, and PySpark on Glue)
- Demonstrated focus on data validation and data quality, including familiarity with frameworks such as Great Expectations, Soda, or custom validation tooling
- Proficiency in Python and SQL, with solid software engineering fundamentals (version control, testing, code review, CI/CD)
- Experience working with AWS services adjacent to the data stack, such as S3, Lambda, Athena, IAM, and CloudWatch
- Comfort working with both structured and semi-structured data formats (Parquet, JSON, Avro)
- Strong communication skills and the ability to translate ambiguous business needs into technical solutions
- A growth mindset — curious, humble, and motivated to solve hard, high-impact problems