Prime Hiring SolutionsRemote
Data Engineer
United States of America
Full Time
3 hours ago
H1B Sponsor Likely
Key skills
Go (Golang)PythonNode.jsAWS LambdaAWS ECS/FargateAWS S3AWS DynamoDBAWS GlueOpenSearch ServerlessAWS IAMAWS CloudWatchAmazon Bedrock Data & AIRetrieval Augmented Generation (RAG)Vector SearchDocument ChunkingContent IndexingKnowledge Ingestion PipelinesTerraformGitLab CI/CDGitOpsInfrastructure as Code (IaC)Secrets ManagementMonitoring & ObservabilityProduction SupportEnterprise Content ManagementAdobe Experience Manager (AEM)Web Scraping/CrawlingAuthenticated Content SourcesEnterprise CMS PlatformsDebuggingGoGolangAILLMRAGAWSGitLab CIServerlessECSFargateLambdaS3DynamoDBIAMCloudWatchGlueBedrockGitLabCI/CD
About this role
Prime Hiring Solutions (PHS) is a workforce solutions partner supporting U.S. clients with high-quality IT, engineering, healthcare, and professional staffing. The Data Engineer will play a crucial role in designing, building, and maintaining enterprise knowledge ingestion pipelines, ensuring the seamless integration of data from various sources.
Responsibilities:
- Design, build, and maintain enterprise knowledge ingestion pipelines
- Develop and enhance web scraping solutions for authenticated content sources (AEM and enterprise portals)
- Implement content extraction, chunking, indexing, and vectorization workflows
- Manage AWS infrastructure using Terraform and Infrastructure as Code (IaC)
- Deploy and support ingestion services using GitLab CI/CD pipelines
- Configure and maintain OpenSearch Serverless, S3, DynamoDB, Lambda, ECS/Fargate, and Glue
- Implement monitoring, logging, alerting, dashboards, and operational runbooks
- Troubleshoot ingestion, infrastructure, and deployment issues across AWS services
- Optimize scalability, reliability, security, and cost of ingestion workloads
- Support RAG-based search and knowledge management solutions using Bedrock and vector search technologies
Requirements:
- Programming
- Go (Golang)
- Python
- Node.js
- AWS Services
- Lambda
- ECS/Fargate
- S3
- DynamoDB
- Glue
- OpenSearch Serverless
- IAM
- CloudWatch
- Bedrock Data & AI
- RAG (Retrieval Augmented Generation)
- Vector Search
- Embeddings
- Document Chunking
- Content Indexing
- Knowledge Ingestion Pipelines
- DevOps & Infrastructure
- Terraform
- GitLab CI/CD
- GitOps
- Infrastructure as Code (IaC)
- Secrets Management
- Monitoring & Observability
- Production Support
- Enterprise Content Management
- Adobe Experience Manager (AEM)
- Web Scraping/Crawling
- Authenticated Content Sources
- Enterprise CMS Platforms
- Experience building enterprise search or knowledge management platforms
- Experience with Amazon Bedrock and LLM-powered applications
- Familiarity with permissions-aware content ingestion
- Strong debugging skills across distributed cloud systems
- Experience supporting production-grade data platforms