Software Engineer – Data Products & Cloud Solutions

The hiring company is confidential. The Senior Software Engineer will help design, build, deploy, and support production data products and services for precision agriculture on the Compass team, spanning backend APIs, data pipelines, cloud infrastructure, databases, and operational tooling.

Responsibilities:

• Architect and implement new AWS and Databricks-based solutions while improving existing Compass services for scalability, reliability, and maintainability
• Design, build, and maintain Python REST APIs, Lambda functions, backend processes, and event-driven services
• Build pipelines that ingest, transform, process, and persist machine, agronomic, geospatial, logistics, and other data
• Build and maintain AWS infrastructure using Terraform
• Design and support PostgreSQL/Aurora databases, including schemas, migrations, application connectivity, and performance
• Build solutions supporting both real-time/event-driven workloads and scheduled batch processing
• Develop and support Databricks integrations and data-processing workflows as needed by Compass products
• Create and maintain CI/CD pipelines using GitHub Actions, including automated testing and application/infrastructure deployment
• Implement and improve logging, metrics, dashboards, monitoring, and alerts using Datadog
• Troubleshoot production issues across APIs, databases, queues, cloud infrastructure, data-processing jobs, and external integrations
• Identify opportunities to improve system performance, scalability, reliability, and cloud cost

Requirements:

• Excellent Python development skills with production software deployment experience
• Strong experience designing and developing REST APIs, backend services, and data-processing applications
• Strong understanding of AWS, including services such as Lambda, API Gateway, IAM, S3, SQS, EventBridge, and RDS/Aurora
• Experience with relational databases, particularly PostgreSQL
• Experience building and maintaining Infrastructure as Code, preferably Terraform
• Experience with CI/CD, automated testing, and Git/GitHub development practices
• Strong troubleshooting skills across application, data, infrastructure, and integration layers
• Ability to independently investigate complex problems, propose solutions, and deliver production-quality implementations
• Strong communication skills and ability to work across engineering, data science, product, and business teams
• Experience improving systems for scalability, reliability, security, performance, and cloud cost efficiency
• Experience with Databricks, Spark, PySpark, SQL, Delta Lake, or similar data platforms
• Experience with event-driven and serverless architectures
• Experience with Docker and containerized applications
• Experience with DBMate, Pydantic, AWS Lambda Powertools, or similar Python frameworks
• Experience with authentication and authorization patterns such as AWS IAM and OAuth/OIDC
• Experience with observability platforms such as Datadog
• Experience processing machine, agricultural, logistics, or other high-volume data
• Experience with GitHub Actions and automated cloud deployments