Senior Data Engineer - Data Insights

Appfire is a company dedicated to empowering teams through innovative software solutions. They are seeking a Senior Data Engineer to enhance their data platform by taking ownership of core infrastructure and driving engineering excellence. The role involves developing scalable data solutions, optimizing data delivery, and implementing AI-driven practices to support company-wide decision-making.

Responsibilities:

• Own the complete solution development lifecycle, from driving initial requirements to building Proof of Concepts (POCs) and Minimum Viable Products (MVPs), and continuously iterating to improve and scale production solutions
• Work closely with internal and external partners to stand up and optimize robust data delivery solutions. Build, manage, and scale complex data pipelines (ETL and reverse ETL) utilizing expert-level Python and Airflow
• Develop standard integration patterns as well as custom data pipelines that interface with atypical APIs to efficiently extend our data delivery capabilities
• Architect and maintain highly efficient, cost-effective scalable data solutions and cloud infrastructure from scratch
• Be the key driver in the evolution of our custom data lake ("firelake" built on AWS/Snowflake), serving as the ultimate subject matter expert on our tech stack and data infrastructure
• Manage Fivetran and custom pipeline operations, proactively handle schema drift, and implement rigorous data quality checks to ensure we have ingested a true and accurate representation of all source systems
• Spearhead AI adoption in our daily engineering workflows to automate Pull Requests, enforce new "definition of done" standards, and accelerate CI/CD, test coverage, and release management
• Drive the integration of AI capabilities directly into our data solutions by interfacing with LLMs and developing intelligent assistants and agents to solve complex business problems
• Stay on the forefront of the rapidly evolving data landscape and drive proof-of-concepts for new tools to ensure we leverage best-in-class technologies to scale quickly
• Implement advanced PII management by leveraging automated data classification techniques, applying masking policies, and enforcing sophisticated row-level and column-level security practices
• Develop and deploy robust monitoring solutions and governance strategies to ensure accurate, secure data is available on time and to the correct audience
• Autonomously lead 3-5 concurrent cross-departmental data projects, driving everything from requirements gathering and UAT to production deployment
• Partner seamlessly with business, analytics, and engineering teams, translating complex technical architectures into relatable concepts to influence stakeholders and align goals
• Champion best practices, systematically reduce technical debt, write clear documentation, and provide both technical and soft-skill mentorship to junior team members

Requirements:

• 5+ years in a data or software engineering role with a deep understanding of the full data lifecycle, modern data warehousing, and agile software development best practices
• 5+ years of experience autonomously building scalable data products, pipelines, and solutions that support company-wide systems and overarching business goals
• Exceptional personal organization and multitasking skills, with the ability to work with minimal supervision while driving 3-5 concurrent cross-departmental projects
• 5+ years of advanced SQL optimization and complex ETL/ELT pipeline development, with extensive, hands-on experience in Snowflake
• 5+ years of advanced Python programming
• 5+ years of experience designing and maintaining AWS cloud environments. Strong, hands-on proficiency with Terraform and Docker is required
• Deep operational experience with Airflow for orchestration, dbt (3+ years) for transformation, and familiarity with managing Fivetran pipelines
• Proven ability to design pragmatic, cost-effective architectures from scratch that prioritize security, scalability, and high performance without over-engineering
• Incredibly strong troubleshooting skills with a ruthless dedication to reducing technical debt, optimizing CI/CD pipelines, and enforcing strict version control and testing standards
• Solid understanding of modern data governance principles, including automated data classification, PII masking, and row/column-level access controls
• Strong written and verbal skills, with a proven ability to translate and simplify complex technical architectures to both engineering peers and non-technical business stakeholders
• A team-oriented mindset with a passion for coaching and providing constructive feedback to junior team members
• Experience in or foundational knowledge of MLOps
• Experience interfacing with LLMs, creating AI agents, or leveraging AI tools to accelerate daily engineering workflows (e.g., automating code reviews, CI/CD enhancements)