Introduction to Data Engineering
How did we get to Data Engineering ?
- Relational data era & traditional Business Intelligence
- Big Data, Hadoop & new storage technologies
- Big Data & new integration methods
- Infrastructures & industrialization (intro)
The place of Data Engineering
- Data value chain
- Roles around the data value chain
- Data Engineering vs Data Scientist
- Data Science & Machine Learning
The Data Engineer
- Basic computer science knowledge required
- Skills in programming languages for each part of the data processing chain
- Knowledge of databases
Data Storage
- Relational databases and SQL (MySQL, Oracle, SQL Server, PostgreSQL),Â
- NoSQL (MongoDB, Elasticsearch…)
- Hadoop (HDFS)