Software Developer & Data Engineer

WISEcode is building a food data platform that turns fragmented food information into consistent, explainable answers. The Software Developer & Data Engineer will help implement and architect its data lakehouse, build ingestion and enrichment pipelines, resolve product identities, and deliver data to downstream products.

Responsibilities:

• Lakehouse Architecture & Implementation: Own end-to-end implementation and collaborate on data/platform architecture alongside our data team
• Source Ingestion: Build and establish patterns for a variety of sources including APIs, text from public sources, and external databases
• Identity & Entity Resolution: Merge the many records we collect for each product, across retailers and dates, into one trustworthy published record. Decide which source wins on each disagreement, and keep every original record so each published value can be traced to its source
• Pipeline Consolidation & Enrichment: Consolidate existing food enrichment pipelines down to a single pipeline in collaboration with our engineering team
• Downstream Connectors: Build robust connectors to feed collapsed food data to WISE Intel and the consumer app
• Operational Discipline & Configuration: Maintain strict fail-fast config discipline (magic values in config files; early loud failures for missing settings)
• Data Quality & Reporting: Establish ingest capabilities and quality reporting for Data Assembly readiness
• Technical Documentation: Write plain, short, dated decision records prior to code implementation

Requirements:

• You bring 3+ years of experience in a similar role
• Native proficiency in Python and SQL
• DuckDB, DuckLake, S3 Parquet
• Postgres, DynamoDB (state management)
• Prefect, Kubernetes/Fargate job runners; familiarity with dbt, Dagster, or Airflow
• S3, ECS, Lambda, EventBridge, SNS, Terraform, IAM
• Lakehouse, analytic/dimension design; Quarto reporting; Medallion architecture (bronze/clean), star schema (food sighting, ingredient, nutrient, provenance)
• github actions
• 3+ years of experience in a similar role
• Proven track record of designing and shipping a lakehouse or data warehouse from scratch, and operating it for at least one year in production
• Demonstrated experience treating data identity and entity resolution as first-class architectural questions
• Experience running batch data pipelines that invoke paid cloud/third-party APIs at scale with strict cost efficiency and budget controls
• Experience pairing daily with other engineers in a fast-paced environment
• Experience making architectural trade-offs between lightweight/embedded query engines (e.g., DuckDB) versus distributed systems (Spark, Databricks, Snowflake)
• Familiarity with data acquisition methods (e.g., web scraping, OCR API integration)