Founding Research Engineer, Model Architecture

Ontologic Intelligence is building a causal reasoning infrastructure for AI systems. The Founding Research Engineer will lead research on decoder-only transformers, focusing on integrating causal variables into model architecture.

Responsibilities:

• Design and implement transformer variants with learned causal variables or latent causal state
• Experiment with modulation of attention, MLP, residual, routing or expert pathways
• Connect persistent external causal graphs or memory with internal model representations
• Implement factual and counterfactual forward passes under controlled interventions
• Train small decoder-only language models from scratch and modify open-weight models
• Build benchmarks and ablations for causal generalisation, intervention consistency and counterfactual reasoning
• Probe whether causal information is genuinely represented and used internally
• Contribute to research and publish strong results

Requirements:

• Strong PyTorch model-engineering skills
• Deep understanding of attention, residual streams, MLP blocks and autoregressive training
• Experience modifying transformer internals, not only prompting or fine-tuning APIs
• Experience training language models at research scale
• Ability to design controlled experiments, baselines and meaningful ablations
• Strong research judgement and disciplined engineering
• Mechanistic interpretability
• causal representation learning
• world models
• memory-augmented transformers
• learned routing
• mixture-of-experts
• slot-based models
• graph neural networks
• FlashAttention
• CUDA
• Triton