Design and implement scalable, cloud-native data pipelines for both batch and streaming use cases using tools such as Azure Data Factory, Databricks, Spark, and Kafka.
Build and maintain multi-dimensional data models optimized for performance, cost, and maintainability.
Integrate upstream data sources (ERP, CRM, external APIs) into core systems with strong attention to data integrity and lineage.
Optimize data flows for storage, processing, and performance using best practices within the Azure ecosystem.
Participate in a shared on-call rotation with Senior and Principal Engineers to ensure data platform reliability.
Resolve production issues, perform root cause analysis, and communicate effectively with stakeholders.
Support operational data tasks (e.g., schema changes, table updates) to maintain agility and consistency.
Contribute to process improvement and operational metrics that enhance system performance and team efficiency.
Partner with product managers, analysts, and engineers to deliver well-architected, scalable data solutions.
Provide peer mentorship and participate in code and design reviews to elevate technical standards.
Influence tooling, architecture, and design decisions that shape the long-term direction of the data platform.
Contribute to documentation, reusability practices, and knowledge sharing across teams.
Requirements
Extensive experience in data engineering or software engineering with a strong data focus.
Advanced proficiency in Python and SQL, with ability to design efficient data transformations and workflows.
Proven experience designing and implementing large-scale ETL pipelines and event-driven architectures (Kafka, Event Hub).
Deep understanding of data warehousing concepts, dimensional modeling, and modern architectures such as data lakes and lakehouses.
Hands-on experience with Azure data services (Data Factory, Databricks, Synapse, Azure Functions).
Expertise working within a major cloud platform (preferably Azure), including architecture and platform design.
Strong understanding of DevOps principles, Git-based version control, and CI/CD pipelines for data systems.
Ability to balance technical excellence with operational ownership, including prioritizing effectively during on-call rotations.
Collaborative mindset with strong communication and problem-solving skills.
Tech Stack
Azure
Cloud
ERP
ETL
Kafka
Python
Spark
SQL
Benefits
18 days PTO*
11 Holidays (8 company recognized & 3 floating holidays)
16 hours per year of paid Volunteer Time Off (VTO)
Competitive Healthcare
401(k) Match: 100% match on the first 3% of your salary, plus 50% match on the next 2%
Parental Leave: 8 weeks 100% paid by AvidXchange**
Discounts on Pet, Home, and Auto insurance
Tuition Reimbursement up to the federal maximum of $5,250***