Key skills
Data engineeringData orchestration tools - dbtData orchestration tools - dagsterData orchestration tools - airflowData warehousesWeb applicationsLogging and event instrumentationMachine learning workflowsAIMachine LearningData EngineeringAnalyticsAirflowdbt
About this role
CoreWeave, the AI Hyperscaler™, acquired Weights & Biases to create the most powerful end-to-end platform to develop, deploy, and iterate AI faster. As a Senior Data Platform Engineer, you will partner with other engineers and data scientists to enhance the data platform while contributing to strategic goals such as migrating data tools and building an internal AI agent.
Responsibilities:
- Partner with other Data Platform Engineers and Data Scientists to expand and maintain our data platform
- Partner with core engineering to improve our product telemetry
- Eventually, join our on-call rotation for the data platform
- Migrate data build tools to improve efficiency and performance
- Build a robust internal data catalog and semantic layer to support self-service analytics
- Build an internal AI agent for addressing common data workflows
- Lay the foundations for internal AI applications built on top of our operational data
Requirements:
- Deep knowledge (practical and theoretical) of data engineering and the accompanying workflows
- History of writing easy-to-follow code and effective documentation for your projects
- Expert knowledge of data orchestration/data pipeline tools (dbt, dagster, airflow, etc)
- Experience working with large and high-demand data warehouses/platforms, multi-PB scale, and extreme business demand for the systems
- Expert knowledge and understanding of web apps and best practices for logging and event instrumentation
- Ability to clearly communicate your ideas to folks across a range of backgrounds and levels of technical knowledge
- If you work well in a self-directed environment and proactively find ways to improve processes and collaborate with team members or engaged users, your initiative will really shine in this role
- You'll love this role if you enjoy connecting with real users daily, helping them solve issues, and understanding good patterns for using our tools. You'll answer questions and requests with a kind, thoughtful tone that makes users feel appreciated and connected to our team
- Theoretical or practical knowledge of machine learning workflows and terminology