Senior Data Engineer for Comprehensive Project
USD 30–250
About the project
Hi, I am looking for an experienced Senior Data Engineer / Data Engineering Architect to design and implement a complete end-to-end, production-style data engineering project that covers a broad range of modern data engineering technologies. This is not intended to be a small ETL project. I want to build a realistic, enterprise-level data platform that demonstrates how the different components of a modern data engineering ecosystem work together. The project should cover the complete data lifecycle: Data Sources → Ingestion → Streaming → Batch Processing → Data Lake → Transformation → Data Warehouse → Data Quality → Orchestration → Analytics → Monitoring → Deployment I also want the freelancer to explain the architecture and technologies clearly throughout the project so that I can understand why each technology is being used, what problem it solves, and how the components integrate with each other. Technical Stack: Python, SQL, Apache Kafka, Spark/PySpark, Databricks, Delta Lake, Airflow, AWS or Azure, Snowflake, dbt, Hive, REST APIs, ETL/ELT, batch and real-time/streaming data pipelines, data lakes/lakehouses, data warehousing, dimensional data modeling, data quality, CI/CD, Git/GitHub, Docker, monitoring, and ideally Terraform.
Skills required
This job is listed on Freelancer.com. AiZity aggregates listings for discovery only and is not the employer. To bid or apply, use the button in the sidebar.