BNSF Railway is one of the largest U.S. railroads transporting freight across the nation. The Data Engineer I/II role involves designing and implementing data streaming solutions, developing robust data pipelines, and collaborating with data scientists and analysts to ensure effective data solutions.
Responsibilities:
- Design and implement data streaming solutions to process real-time data efficiently
- Develop and maintain robust data pipelines for both real time and batch processing including data extraction, transformation and ETL processes to ensure seamless data flow across systems
- Create and manage large scale data warehouses to ensure efficient data access and retrieval in support of business intelligence and analytics needs
- Develop certified datasets by integrating data from diverse data sources to ensure data accuracy, consistency and reliability
- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver effective data solutions
- Optimize data workflows, monitor data pipelines and troubleshoot issues to ensure high performance, reliable and scalable data flows
- Ensure data security and compliance with industry standards and regulations
- Stay current with industry trends and advancements in data engineering, data science and machine learning
- Write efficient and well documented code ensuring unit tests documentation for all data solutions
- Implement data engineering solutions into data platforms or production systems by leading the solution design, development, testing, and monitoring
Requirements:
- Authorized to work in the US
- Minimum of 2 years of experience in software engineering, data engineering, database management, or relevant experience
- Proficiency in programming languages such as Python, SQL, Java
- Strong understanding of data warehousing concepts including dimensional modeling and star schema
- Experience with ETL processes and tools including cloud based solutions
- Familiarity with database management systems (e.g., SQL, NoSQL) and modern lakehouse technologies (eg. Apache Iceberg, Delta Lake, Apache Trino)
- Experience with data streaming technologies (e.g., Apache Kafka, AWS Kinesis, Apache Flink and Apache Spark)
- Experience in data engineering, with a focus on data streaming, data pipelines, data orchestration tools, datamart creation, and certified datasets
- Excellent problem-solving skills and attention to detail
- Strong communication and collaboration skills
- Intellectual curiosity and ability to thrive in a team environment that is constantly changing
- Demonstrated ability to learn new technical concepts and to adapt to new technologies quickly
- Ability to communicate in a clear, concise, professional oral or written manner, to be understood by technical and non-technical colleagues
- Able to work now and in the future without BNSF's assistance (whether monetary, through sponsorship, or otherwise) in obtaining, maintaining, or extending employment authorization (including H-1B, STEM OPT/CPT, or TN nonimmigrant status)
- Bachelor's degree in Computer Science, Information Technology or a related field
- Experience in DevSecOps standards, source control systems and tools like Jenkins, GitHub
- Prior experience with Data Engineering tools and architectures
- Experience in Rail, Shipping, Airline, Logistics, Warehousing, Supply Chain, or other Transportation industries or experience in a High-Tech industry
- Experience working in an Agile environment (Scrum, Kanban, SAFe)
- Experience with cloud platforms
- Proficiency in using Jupyter for collaborative data analysis
- Experience with open-source technologies and tools
- Have knowledge of geospatial data transformation and spatial libraries