Key skills
AirflowAzureBigQueryCloudETLGoogle Cloud PlatformJavaPythonScalaSparkSQLAIData LakeDatabricksFivetranGCPGoogle Cloud
About this role
Role Overview
- Leverage established patterns and establish new patterns as required to rapidly onboard new data sets into the data platform
- Establish repeatable, automated data management processes for managing sensitive information
- Own deliverables while also collaborating with and supporting others in a team setting including conducting and participating in design and code reviews
- Collaborate with product on priorities and approach and with other engineering and AI teams who would consume raw data
Requirements
- 2+ years of engineering experience with data warehouses, data lakes or software engineering with a data emphasis
- Strong SQL knowledge
- Proficient in Scala, Python, or Java
- Well versed in Data Lake & Delta Lake Concepts
- Understanding of data security best practices
- Well versed in Databricks/BigQuery/Palantir usage in building enterprise data platforms
- Familiarity with implementing end-to-end data pipeline and Ontology development using Palantir platform is a plus
- Familiarity with data orchestration technologies like Airflow, Dagster, or Fivetran
- Experience with cloud environments; (GCP, Azure) a definite plus
- Familiar with DevOps process
- Experience creating and sharing standards, best practices, documentation, and reference examples for data warehouse, integration/ETL systems
- Apply a disciplined approach to testing software and data
- Experience working with APIs
- Well versed in common data frameworks such as Spark & Polars
- Undergraduate degree in computer science or related discipline preferred or equivalent experience along with a demonstrated desire for continuing education and improvement.
Tech Stack
- Airflow
- Azure
- BigQuery
- Cloud
- ETL
- Google Cloud Platform
- Java
- Python
- Scala
- Spark
- SQL
Benefits
- Competitive compensation
- Generous vacation policy, paid holidays, and paid sick time
- Medical Insurance, Dental Insurance, and Vision Insurance (employee-paid)
- Company-paid Short-Term and Long-Term Disability Insurance
- Company-paid Group Life insurance
- Company-paid Employee Assistance Program (EAP) and Calm App subscription
- Employee-paid Pet Insurance and optional supplemental insurance coverage
- Vested 401(k) with company match and financial wellness programs
- Flexible Spending Account (FSA), Health Savings Account (HSA) and commuter benefits options
- Paid maternity leave, paid paternity leave, and fertility benefits
- Career growth and learning opportunities
- …and so much more!