Senior Data Engineer

Requisition #: 1478

Job Title: Senior Data Engineer

Location: Washington D.C.

Clearance Level: Top Secret

SUMMARY

The Senior Data Engineer will support the development, maintenance, and optimization of enterprise data systems and pipelines that power a Private Sector Portal (PSP) and related data services. This role focuses on designing scalable ETL processes, automating data workflows, and ensuring high-quality, standardized data across the organization. The engineer will work closely with relationship management teams, developers, and enterprise stakeholders to gather technical requirements, troubleshoot issues, and deliver reliable data solutions.

As a direct employee of Agile Defense, you would receive a benefit package that includes health/dental/vision insurance coverage, 401K with company match, PTO & paid holidays, and annual tuition/training assistance. For more information, please visit our website.

JOB DUTIES AND RESPONSIBILITIES

· Update and maintain the information residing on the Private Sector Portal (PSP) tool, using automation when appropriate.

· Assist the relationship management team in identifying and documenting technical requirements around the PSP system.

· Develop guidance around appropriate use of the PSP system and ensure consistent data standards across the enterprise.

· Provide expertise and customer service to HQ/field users.

· Maintain the PSP system, to include auditing content for missing or inappropriate metadata, reporting technical user problems to the contractor.

· Ensure the scheduling, processing, and documentation of data delivery and implementation, using automation when appropriate

· Design, implement, automate, and maintain Enterprise level of scalable Extract, Transform and Load (ETL) data pipelines and data analysis pipelines.

· Build and maintain robust data processing, enriching, and analysis pipelines that aggregate and normalize large datasets across diverse sources, including sources from web services. data from disparate sources.

· Automate and optimize database and data workflows for best performance and efficiency.

· Identify and propose emerging technologies, automation or alternative solutions to efficiently improve support for developers and application stakeholders.

· Develop and maintain documentation, inline comments, confluence and stand-alone documents, wherever necessary for the readability of the software design and code.

· Ability to focus on producing results efficiently.

· Ability to work independently and coordinate with team members.

QUALIFICATIONS

Education, Background, and Years of Experience

· Bachelor's degree from an accredited university or college in a quantitative field (i.e. Statistics or Operations Research) or technical field (i.e. Computer Science or Engineering).

· Educational requirement may be waived if the candidate has eight (8) or more years of experience.

· Minimum 10 years of relevant experience.

· Four (4) years of technical writing experience for the Federal Government.

· Four (4) years of Exceptional proficiency in writing and editing skills using MS Office.

· Minimum of 2-4 years of work experience in a data analytics or data engineering role

ADDITIONAL SKILLS & QUALIFICATIONS Required Skills

· Proficiency with a scripting language (e.g. Python, R, Perl, bash) and advanced SQL.

· Experience with business visualization tools (e.g. Looker, Tableau, Microsoft PowerBI).

· Experience with HTML, CSS, Bootstrap/Reactstrap, and JavaScript frameworks such as React.

· Ability to explain technical concepts to non-technical people.

· Experience with data warehousing.

· Experience with AWS, Google Cloud ecosystem (e.g. BigQuery, Redshift).

· Experience with designing, implementing and maintaining big data processing, normalizing, and enriching pipelines in language such as Python, PySpark, or Spark.

· Experience with architecting, designing and building modern medallion ETL processes for data lake, data warehouse or lakehouse.

· Experience with building and Knowledge of automating data pipeline and workflow on Apache Niagra Files (NiFi) or Databricks for ease of maintenance.

· Experience with implementing automated data Cl/CD pipelines with Continuous Integration/Continuous Delivery (CI/CD).

· Experience with dataframe processing with filtering conditions.

· Experience with organizing data version history to optimize processing and retrieval performance.

· Experience with data indexing for robust searching and querying.

· Strong verbal and written communication skills to effectively collaborate and integrate with enterprise partners and customers.

· Excellent at troubleshooting, debugging and researching.

· Ability to focus and a desire to learn.

· Knowledge of PostgreSQL is preferred.

WORKING CONDITIONS

Full time on-site with client in office setting.