Aya HealthcareRemote
Data Engineer
United States of America
Full Time
2 hours ago
$100,000 USD
H1B Sponsor Likely
Key skills
Azure cloud servicesCloud data architectureETL/ELT pipelinesSQLPythonBig data frameworksDocumentation managementAnalytical problem-solvingJavaScalaETLELTData EngineeringData LakeSparkApache SparkKafkaApache KafkaHadoopAzureCloud Storage
About this role
Aya Healthcare is a rapidly growing workforce solutions provider in the healthcare industry, known for its commitment to employee satisfaction and innovative thinking. The Data Engineer will design, develop, and implement scalable data infrastructure and pipelines for client integrations, ensuring data quality and compliance while collaborating with client engineering teams.
Responsibilities:
- Monitor and troubleshoot data pipeline and warehouse quality and performance issues, ensuring smooth operation and proactively addressing potential bottlenecks
- Design, build, and maintain scalable ETL/ELT pipelines to transfer data to and from clients from diverse sources, such as APIs, cloud storage, and databases
- Collaborate with client engineering teams to understand their data structures, identify source systems, and contribute to technical integration requirements
- Stay updated with industry trends and advancements in data engineering and bring innovative ideas to continuously improve our data processes
- Ensure data security, privacy, and compliance by implementing appropriate Azure policies, procedures, and encryption methods
- Monitor and optimize the performance, scalability, and cost-efficiency of data storage and processing solutions on Azure
Requirements:
- Bachelor's degree in computer science, information systems, or a related field
- 2+ years of experience in cloud data architecture, design and data governance
- Strong experience with Azure cloud services, such as Azure Data Lake Storage, Azure Data Factory, and Azure Synapse
- Proficiency in programming languages, such as SQL, Python, Scala, or Java
- Experience with big data processing frameworks, such as Apache Spark, Hadoop, or Apache Kafka on Azure
- Knowledge of various data storage solutions, such as relational databases, etc
- Creating & Managing documentation on methodologies, tools and techniques
- Strong analytical problem-solving and the ability to explain complex technical designs to non-technical stakeholders
- Willing and able to work irregular hours as needed