Data Engineer

Artlogic is a leading technology platform for the art world, empowering galleries and art professionals to streamline their operations. The Junior Data Engineer will be responsible for building and executing ETL pipelines to facilitate client data migration and ensure data integrity during the onboarding process.

Responsibilities:

• Execute End-to-End Migrations: Learn to navigate the full migration lifecycle, moving data from diverse client environments into Artlogic
• Extraction: Use scripts and tools to pull data from external platforms (e.g., FileMaker, legacy SQL databases)
• Transformation: Write clean, maintainable code to map, clean, and transform "messy" legacy data to fit the Artlogic schema
• QA & Validation: Perform rigorous data validation and unit testing to ensure 100% accuracy and data integrity before go-live
• Code Contribution: Assist in developing and maintaining our internal migration frameworks and R&D tools
• Pipeline Optimization: Identify repetitive tasks in your daily workflow and work with senior engineers to automate them
• Systems Analysis: Research unknown database structures to determine the most efficient extraction methods
• Cross-functional Teamwork: Partner with Client Liaison Project Managers to understand client-specific data nuances
• Documentation: Document migration mappings and technical edge cases to ensure knowledge is shared across the team
• Iterative Delivery: Adapt migration scripts quickly as project requirements evolve during the onboarding phase
• AI Recommendation Verification and Data Mapping: Review and verify AI (Claude) recommendations for column mappings during cloud data migration, including confirming that columns fit perfectly or identifying where data requires manipulation or concatenation
• Data Transformation and Cloud Push: Perform final data transformation steps using basic Python, Pandas, and Reax code to prepare and push relational data from Excel into the cloud environment

Requirements:

• Proficiency in at least one object-oriented language—Python is preferred, but we also value strong logic in C#
• Solid understanding of relational databases (Joins, Keys, Indexes) and how to manipulate data structures
• Ability to explain technical data hurdles to non-technical Project Managers
• An obsession with data accuracy; you catch the 'edge case' before it becomes a bug
• Enjoy the 'detective work' of figuring out how a legacy system was built and how to extract its value
• Experience with ETL tools, pandas (Python), or regular expressions (Regex)
• Familiarity with REST APIs and how to handle JSON/XML data
• Previous exposure to FileMaker (Claris) or NoSQL environments
• A high-level understanding of how back-end data interacts with front-end UIs