INSPYR SolutionsRemote
Lead Data Engineer
United States of America
Full Time
2 hours ago
$150,000 USD
H1B Sponsor Likely
Key skills
DatabricksPySparkGCPData lake architecturePythonSQLAirflowData modelingStakeholder engagementAIData EngineeringData LakeBigQuerydbtAWSCloud StorageGCS
About this role
INSPYR Solutions is a national expert in delivering flexible technology and talent solutions. They are seeking a Lead Data Engineer to design and build a next-generation data platform, focusing on modernizing data architecture and collaborating with technical teams and business stakeholders to implement scalable solutions.
Responsibilities:
- Lead the design and implementation of scalable data pipelines and data platforms using modern lakehouse architecture
- Build and optimize production-grade PySpark pipelines in Databricks
- Design and implement canonical data models across multiple data sources
- Apply medallion architecture (bronze/silver/gold layers) for structured and unstructured data
- Drive data quality improvements in a complex, mixed-format environment (JSON, CSV, XML/DDEX)
- Partner with business stakeholders to run workshops, gather requirements, and translate needs into technical solutions
- Architect solutions leveraging GCP (BigQuery, Dataproc, Cloud Storage) and orchestration with Airflow (Astro)
- Mentor and guide a team of data engineers, including contractors
- Contribute to best practices around scalability, reliability, and performance
- Leverage modern tools (including AI) to improve engineering productivity
Requirements:
- 6+ years of data engineering experience, including 2+ years in a lead or senior-lead capacity
- Deep, production-level experience with Databricks and PySpark (not just PoC work)
- Strong experience with GCP, including: BigQuery, Dataproc, Cloud Storage (GCS)
- Experience designing data lake / lakehouse architectures (GCS as system-of-record is a plus)
- Advanced data modeling expertise, including: Dimensional modeling, Canonical/domain modeling, Entity resolution
- Strong proficiency in Python and SQL, with a focus on production-quality, reliable code
- Experience with Airflow (or Astro managed Airflow) for orchestration
- Proven ability to work directly with business stakeholders and lead requirements-gathering sessions
- Experience delivering end-to-end data platform implementations
- Experience with dbt (especially within medallion architectures)
- Background in music, media, digital rights, or royalties
- Exposure to AWS environments
- Experience working in startup or consulting environments