面向数据工程师的大数据

Big Data for Data Engineers

Build Your Data Engineering Skills。Learn how to tame the big data beast with the most popular tools assisted by top-notch practitioners

Yandex

专项课程

分享

  • 分类: 计算机
  • 平台: Coursera
  • 语言: 英语

本专项课程介绍

This specialization is made for people working with data (either small or big). If you are a Data Analyst, Data Scientist, Data Engineer or Data Architect (or you want to become one) — don’t miss the opportunity to expand your knowledge and skills in the field of data engineering and data analysis on the large scale.

In four concise courses you will learn the basics of Hadoop, MapReduce, Spark, methods of offline data processing for warehousing, real-time data processing and large-scale machine learning. And Capstone project for you to build and deploy your own Big Data Service (make your portfolio even more competitive).

Over the course of the specialization, you will complete progressively harder programming assignments (mostly in Python). Make sure, you have some experience in it. This course will master your skills in designing solutions for common Big Data tasks:

– creating batch and real-time data processing pipelines,
– doing machine learning at scale,
– deploying machine learning models into a production environment — and much more!

Join some of best hands-on big data professionals, who know, their job inside-out, to learn the basics, as well as some tricks of the trade, from them.

Special thanks to: Prof. Mikhail Roytberg (APT dept., MIPT), Oleg Sukhoroslov (PhD, Senior Researcher, IITP RAS), Oleg Ivchenko (APT dept., MIPT), Pavel Akhtyamov (APT dept., MIPT), Vladimir Kuznetsov, Asya Roitberg, Eugene Baulin, Marina Sudarikova.

项目概览

Are you ready to close the loop on your Big Data skills? Do you want to apply all your knowledge you got from the previous courses in practice? Finally, in the Capstone project, you will integrate all the knowledge acquired earlier to build a real application leveraging the power of Big Data.

You will be given a task to combine data from different sources of different types (static distributed dataset, streaming data, SQL or NoSQL storage). Combined, this data will be used to build a predictive model for a financial market (as an example). First, you design a system from scratch and share it with your peers to get valuable feedback. Second, you can make it public, so get ready to receive the feedback from your service users. Real-world experience without any 3D-glasses or mock interviews.

第 1 门课程

Big Data Essentials: HDFS, MapReduce and Spark RDD

计划开课班次:Jan 29
课程学习时间6 weeks of study, 6-8 hours/week

课程概述
Have you ever heard about such technologies as HDFS, MapReduce, Spark? Always wanted to learn these new tools but missed concise starting material? Don’t miss this course either!

In this 6-week course you will:
– learn some basic technologies of the modern Big Data landscape, namely: HDFS, MapReduce and Spark;
– be guided both through systems internals and their applications;
– learn about distributed file systems, why they exist and what function they serve;
– grasp the MapReduce framework, a workhorse for many modern Big Data applications;
– apply the framework to process texts and solve sample business cases;
– learn about Spark, the next-generation computational framework;
– build a strong understanding of Spark basic concepts;
– develop skills to apply these tools to creating solutions in finance, social networks, telecommunications and many other fields.

Your learning experience will be as close to real life as possible with the chance to evaluate your practical assignments on a real cluster. No mocking, a friendly considerate atmosphere to make the process of your learning smooth and enjoyable.

Get ready to work with real datasets alongside with real masters!

Special thanks to:
– Prof. Mikhail Roytberg, APT dept., MIPT, who was the initial reviewer of the project, the supervisor and mentor of half of the BigData team. He was the one, who helped to get this show on the road.
– Oleg Sukhoroslov (PhD, Senior Researcher at IITP RAS), who has been teaching MapReduce, Hadoop and friends since 2008. Now he is leading the infrastructure team.
– Oleg Ivchenko (PhD student APT dept., MIPT), Pavel Akhtyamov (MSc. student at APT dept., MIPT) and Vladimir Kuznetsov (Assistant at P.G. Demidov Yaroslavl State University), superbrains who have developed and now maintain the infrastructure used for practical assignments in this course.
– Asya Roitberg, Eugene Baulin, Marina Sudarikova. These people never sleep to babysit this course day and night, to make your learning experience productive, smooth and exciting.

第 2 门课程

Big Data Analysis: Hive, Spark SQL, DataFrames and GraphFrames

计划开课班次:Jan 29
课程学习时间6 weeks of study, 6-8 hours/week

课程概述
No doubt working with huge data volumes is hard, but to move a mountain, you have to deal with a lot of small stones. But why strain yourself? Using Mapreduce and Spark you tackle the issue partially, thus leaving some space for high-level tools. Stop struggling to make your big data workflow productive and efficient, make use of the tools we are offering you.

This course will teach you how to:
– Warehouse your data efficiently using Hive, Spark SQL and Spark DataFframes.
– Work with large graphs, such as social graphs or networks.
– Optimize your Spark applications for maximum performance.

Precisely, you will master your knowledge in:
– Writing and executing Hive & Spark SQL queries;
– Reasoning how the queries are translated into actual execution primitives (be it MapReduce jobs or Spark transformations);
– Organizing your data in Hive to optimize disk space usage and execution times;
– Constructing Spark DataFrames and using them to write ad-hoc analytical jobs easily;
– Processing large graphs with Spark GraphFrames;
– Debugging, profiling and optimizing Spark application performance.

Still in doubt? Check this out. Become a data ninja by taking this course!

Special thanks to:
– Prof. Mikhail Roytberg, APT dept., MIPT, who was the initial reviewer of the project, the supervisor and mentor of half of the BigData team. He was the one, who helped to get this show on the road.
– Oleg Sukhoroslov (PhD, Senior Researcher at IITP RAS), who has been teaching MapReduce, Hadoop and friends since 2008. Now he is leading the infrastructure team.
– Oleg Ivchenko (PhD student APT dept., MIPT), Pavel Akhtyamov (MSc. student at APT dept., MIPT) and Vladimir Kuznetsov (Assistant at P.G. Demidov Yaroslavl State University), superbrains who have developed and now maintain the infrastructure used for practical assignments in this course.
– Asya Roitberg, Eugene Baulin, Marina Sudarikova. These people never sleep to babysit this course day and night, to make your learning experience productive, smooth and exciting.

第 3 门课程

Big Data Applications: Machine Learning at Scale

计划开课班次:Jan 29
课程学习时间5 weeks of study, 6-8 hours/week

课程概述
Machine learning is transforming the world around us. To become successful, you’d better know what kinds of problems can be solved with machine learning, and how they can be solved. Don’t know where to start? The answer is one button away.

During this course you will:
– Identify practical problems which can be solved with machine learning
– Build, tune and apply linear models with Spark MLLib
– Understand methods of text processing
– Fit decision trees and boost them with ensemble learning
– Construct your own recommender system.

As a practical assignment, you will
– build and apply linear models for classification and regression tasks;
– learn how to work with texts;
– automatically construct decision trees and improve their performance with ensemble learning;
– finally, you will build your own recommender system!

With these skills, you will be able to tackle many practical machine learning tasks.

We provide the tools, you choose the place of application to make this world of machines more intelligent.

Special thanks to:
– Prof. Mikhail Roytberg, APT dept., MIPT, who was the initial reviewer of the project, the supervisor and mentor of half of the BigData team. He was the one, who helped to get this show on the road.
– Oleg Sukhoroslov (PhD, Senior Researcher at IITP RAS), who has been teaching MapReduce, Hadoop and friends since 2008. Now he is leading the infrastructure team.
– Oleg Ivchenko (PhD student APT dept., MIPT), Pavel Akhtyamov (MSc. student at APT dept., MIPT) and Vladimir Kuznetsov (Assistant at P.G. Demidov Yaroslavl State University), superbrains who have developed and now maintain the infrastructure used for practical assignments in this course.
– Asya Roitberg, Eugene Baulin, Marina Sudarikova. These people never sleep to babysit this course day and night, to make your learning experience productive, smooth and exciting.

第 4 门课程

Big Data Applications: Real-Time Streaming

于 November 2017 开始
课程学习时间4 weeks of study, 6-8 hours/week

课程概述
There is a significant number of tasks when we need not just to process an enormous volume of data but to process it as quickly as possible. Delays in tsunami prediction can cost people’s lives. Delays in traffic jam prediction cost extra time. Advertisements based on the recent users’ activity are ten times more popular.

However, stream processing techniques alone are not enough to create a complete real-time system. For example to create a recommendation system we need to have a storage that allows to store and fetch data for a user with minimal latency. These databases should be able to store hundreds of terabytes of data, handle billions of requests per day and have a 100% uptime. NoSQL databases are commonly used to solve this challenging problem.

After you finish this course, you will master stream processing systems and NoSQL databases. You will also learn how to use such popular and powerful systems as Kafka, Cassandra and Redis.

To get the most out of this course, you need to know Hadoop and Hive. You should also have a working knowledge of Spark, Spark SQL and Python.

Do you want to learn how to build Big Data applications that can withstand modern challenges? Jump right in!

第 5 门课程

Big Data Services: Capstone Project

于 December 2017 开始

课程概述
Are you ready to close the loop on your Big Data skills? Do you want to apply all your knowledge you got from the previous courses in practice? Finally, in the Capstone project, you will integrate all the knowledge acquired earlier to build a real application leveraging the power of Big Data.

You will be given a task to combine data from different sources of different types (static distributed dataset, streaming data, SQL or NoSQL storage). Combined, this data will be used to build a predictive model for a financial market (as an example). First, you design a system from scratch and share it with your peers to get valuable feedback. Second, you can make it public, so get ready to receive the feedback from your service users. Real-world experience without any 3G-glasses or mock interviews.

Udemy
声明:MOOC中国发布之课程均源自下列机构,版权均归他们所有。本站仅作报道收录并尊重其著作权益,感谢他们对MOOC事业做出的贡献!(排名不分先后)
  • Coursera
  • edX
  • OpenLearning
  • FutureLearn
  • iversity
  • Udacity
  • NovoEd
  • Canvas
  • Open2Study
  • Google
  • ewant
  • FUN
  • IOC-Athlete-MOOC
  • World-Science-U
  • Codecademy
  • CourseSites
  • opencourseworld
  • ShareCourse
  • gacco
  • MiriadaX
  • JANUX
  • openhpi
  • Stanford-Open-Edx
  • 网易云课堂
  • 中国大学MOOC
  • 学堂在线
  • 顶你学堂
  • 华文慕课
  • 好大学在线CnMooc
  • 以及更多...

Copyright © 2008-2015 MOOC.CN 慕课改变你,你改变世界