Key skills
AWSETLPandasPostgresPythonReactTypeScriptFastAPIS3PostgreSQLCaching
About this role
Role Overview
- Work on building and evolving a data reconciliation platform.
- Improve performance, scalability, and user experience for features such as reconciliation, filters, pagination, export, and AWS integrations.
- Enhance the ETL pipeline with a focus on performance (filtering, joins, deduplication, aggregation).
- Optimize read/write of large files (CSV, Parquet, GZIP).
- Design and operate persistence in PostgreSQL (schema, indexes, queries).
- Implement caching, server-side pagination, and sorting for large datasets.
- Ensure processing stability and scalability.
Requirements
- Backend architecture and stack in Python (FastAPI) with an ETL pipeline and data reconciliation.
- Large-scale processing using Polars, pandas, or DuckDB.
- Persistence and metadata in PostgreSQL (jobs, manifests, history, parameters).
- File and result storage in AWS S3.
- Frontend in React + TypeScript with data grids, filters, and server-side pagination.
- Experience tuning ETL and handling large-scale data.
- Data modeling and optimization in PostgreSQL (indexes, EXPLAIN, VACUUM/ANALYZE).
- Familiarity with job-oriented and queue-based architecture patterns.
Tech Stack
- AWS
- ETL
- Pandas
- Postgres
- Python
- React
- TypeScript
Benefits
- 🍽 Meal voucher and/or food allowance (iFood Benefícios)
- 🏥 Health insurance (Amil)
- 💼 Life insurance
- 😁 Dental insurance (Amil)
- 🚗 Fuel allowance
- 🏋 Gympass/Wellhub: Access to gyms with a variety of training modalities.
- 🐶 Pet health insurance
- 👶🏻 Childcare assistance
- 💳 Lincard: Partnership club offering discounts of up to 60% at over 4,000 establishments in Brazil.