This course is designed to equip students with practical skills and knowledge in data engineering, focusing on the application of data processing, storage, and analysis techniques in real-world scenarios. Students will learn to design, build, and manage data pipelines, integrate data from various sources, and optimize data systems for performance and scalability. The course emphasizes hands-on experience with industry-standard tools and technologies, enabling students to address complex data challenges and support data-driven decision-making in professional environments. By the end of the course, students will be proficient in the practical aspects of data engineering, prepared to implement robust data solutions in diverse organizational settings.

This course provides a comprehensive exploration of the practical aspects of data engineering, focusing on the design, implementation, and management of data pipelines and systems. Students will gain hands-on experience with data integration, processing, and storage technologies, learning how to work with both structured and unstructured data. The curriculum covers key topics such as data warehousing, ETL (Extract, Transform, Load) processes, big data technologies, and performance optimization. Through real-world projects, students will develop the skills necessary to build scalable and efficient data infrastructures that support data-driven decision-making in various industries.

Resources: -Lecture Notes and Codes - Elasticsearch in Action, Madhusudhan Konda, Manning Publications, Second edition or newer - Data Pipelines with Apache Airflow, Bas Harenslak & Julian de Ruiter, Manning Publications, latest edition Textbook: Data Engineering with Python, Paul Crickard, Packt Publishing, First edition or newer.