Data Pipeline Integration for Food Databases

About the project

Summary Need a Data Pipeline Engineer (Python / ETL / PostgreSQL) We are looking for an experienced Data Pipeline Engineer to integrate multiple food databases into an existing production backend. Important: This is not a greenfield project. The backend, database schema, APIs, and food intelligence engine have already been built. Your role is to build and maintain the ingestion pipelines that feed the existing system. Current Backend Status * Existing PostgreSQL database schema * Existing backend APIs * Existing food scoring engine * Existing product model and normalization framework * Existing infrastructure for product ingestion Scope of Work Build reliable ingestion pipelines to import, clean, normalize, validate, and synchronize data from the following sources: * Open Food Facts * USDA FoodData Central * One Latin American food database * One Chinese food database * One Southeast Asian food database Your responsibilities include: * Downloading data from APIs or bulk datasets * Parsing and transforming source data * Mapping source fields into our existing schema * Deduplicating products * Importing and linking product images where available * Building reliable, resumable ETL pipelines * Creating incremental update jobs * Producing validation and import reports * Documenting the ingestion process Required Skills * Python * PostgreSQL * SQL * ETL / ELT pipeline development * Data modeling * REST APIs * JSON / CSV / XML processing * Data validation and deduplication * Git * Docker (preferred) Experience with food datasets, product catalogs, or large-scale data ingestion is a strong plus. Long-Term Opportunity This engagement represents the first phase of a much larger data infrastructure initiative. There is a strong possibility of a long-term collaboration as we continue expanding our global food intelligence platform by integrating additional regional and commercial data sources. We are looking for someone who can grow with the project and contribute to future phases. When Applying Please include the following in your initial proposal: 1. Your proposed technical approach for integrating these data sources into our existing backend. 2. Relevant ETL or data pipeline projects you have completed. 3. Your proposed implementation timeline, including major milestones. 4. Your proposed fixed-price or milestone-based quote for completing this scope of work. Please include your proposed timeline and quote in your initial proposal. We are not looking to go back and forth to obtain this information. Applications that do not include a proposed approach, timeline, and quote may not be considered. We’re looking for someone who can begin immediately and deliver a robust, maintainable pipeline that can be extended with additional data sources in future phases.

Skills required