This course is part of the Learn SQL Basics for Data Science Specialization

4.5
stars
574 ratings

Brooke Wenig +1 more instructor

36,798 already enrolled

Offered By

Learn SQL Basics for Data Science SpecializationUniversity of California, Davis

About this Course

34,278 recent views

This course is all about big data. It’s for students with SQL experience that want to take the next step on their data journey by learning distributed computing using Apache Spark. Students will gain a thorough understanding of this open-source standard for working with large datasets. Students will gain an understanding of the fundamentals of data analysis using SQL on Spark, setting the foundation for how to combine data with advanced analytics at scale and in production environments. The four modules build on one another and by the end of the course you will understand: the Spark architecture, queries within Spark, common ways to optimize Spark SQL, and how to build reliable data pipelines. 
The first module introduces Spark and the Databricks environment including how Spark distributes computation and Spark SQL. Module 2 covers the core concepts of Spark such as storage vs. compute, caching, partitions, and troubleshooting performance issues via the Spark UI. It also covers new features in Apache Spark 3.x such as Adaptive Query Execution. The third module focuses on Engineering Data Pipelines including connecting to databases, schemas and data types, file formats, and writing reliable data. The final module covers data lakes, data warehouses, and lakehouses. Students build production grade data pipelines by combining Spark with the open-source project Delta Lake. By the end of this course, students will hone their SQL and distributed computing skills to become more adept at advanced analysis and to set the stage for transitioning to more advanced analytics as Data Scientists.

Flexible deadlines

Reset deadlines in accordance to your schedule.

Shareable Certificate

Earn a Certificate upon completion

100% online

Start instantly and learn at your own schedule.

Coursera Labs

Includes hands on learning projects.

Learn more about Coursera Labs

Course 3 of 4 in the

Learn SQL Basics for Data Science Specialization

Intermediate Level

Approx. 13 hours to complete

English

Could your company benefit from training employees on in-demand skills?

Try Coursera for Business

What you will learn

Use the collaborative Databricks workspace to write scalable Spark SQL code that executes against a cluster of machines
Inspect the Spark UI to analyze query performance and identify bottlenecks
Create an end-to-end pipeline that reads data, transforms it, and saves the result
Build a medallion (bronze, silver, gold) lakehouse architecture with Delta Lake to ensure the reliability, scalability, and performance of your data

Skills you will gain

Data Science
Apache Spark
Delta Lake
SQL

Flexible deadlines

Reset deadlines in accordance to your schedule.

Shareable Certificate

Earn a Certificate upon completion

100% online

Start instantly and learn at your own schedule.

Coursera Labs

Includes hands on learning projects.

Learn more about Coursera Labs

Course 3 of 4 in the

Learn SQL Basics for Data Science Specialization

Intermediate Level

Approx. 13 hours to complete

English

Could your company benefit from training employees on in-demand skills?

Try Coursera for Business

Instructors

Instructor rating

4.68/5 (123 Ratings)

Brooke Wenig

Machine Learning Practice Lead at Databricks

Continuing and Professional Education

36,798 Learners

1 Course

Conor Murphy

Lead Data Scientist at Databricks

Continuing and Professional Education

36,798 Learners

1 Course

Offered by

University of California, Davis

See how employees at top companies are mastering in-demand skills

Learn more about Coursera for Business

Syllabus - What you will learn from this course

Content Rating87%(1,638 ratings)

Week1

Week 1

3 hours to complete

Introduction to Spark

3 hours to complete

6 videos (Total 43 min), 3 readings, 2 quizzes

Week2

Week 2

3 hours to complete

Spark Core Concepts

3 hours to complete

6 videos (Total 36 min), 2 readings, 2 quizzes

Week3

Week 3

4 hours to complete

Engineering Data Pipelines

In this module, you will be able to identify and discuss the general demands of data applications. You'll be able to access data in a variety of formats and compare and contrast the tradeoffs between these formats. You will explore and examine semi-structured JSON data (common in big data environments) as well as schemas and parallel data writes. You will be able to create an end-to-end pipeline that reads data, transforms it, and saves the result.

4 hours to complete

7 videos (Total 62 min), 2 readings, 2 quizzes

Week4

Week 4

4 hours to complete

Data Lakes, Warehouses and Lakehouses

In this module, you will identify the key characteristics of data lakes, data warehouses, and lakehouses. Lakehouses combine the scalability and low-cost storage of data lakes with the speed and ACID transactional guarantees of data warehouses. You will build a production grade lakehouse by combining Spark with the open-source project, Delta Lake. Whoever said time travel isn't possible hasn't been to a lakehouse!

4 hours to complete

8 videos (Total 52 min), 2 readings, 3 quizzes

Reviews

4.5

148 reviews

5 stars
66.08%
4 stars
23.13%
3 stars
6.60%
2 stars
1.91%
1 star
2.26%

TOP REVIEWS FROM DISTRIBUTED COMPUTING WITH SPARK SQL

by JRMar 17, 2022

A good introduction to Spark SQL.

A pity that the course assignment was not very challenging but more focused on understanding of Spark itself.

by ITFeb 22, 2020

Great introduction to Spark SQL and ML Flow. I love that they give extra resources if you want to learn more. It was a fun learning journey.

by DFeb 11, 2020

This has been an amazing course. What is worth mentioning is how the content was delivered. Nice hands on. Highly recommended for anyone who is new to Spark

by ACMay 8, 2021

I would like to thank all the instructors of the course and the Coursera team for preparing such a nice and very well understandable course for beginners.

View all reviews

About the Learn SQL Basics for Data Science Specialization

This Specialization is intended for a learner with no previous coding experience seeking to develop SQL query fluency. Through four progressively more difficult SQL projects with data science applications, you will cover topics such as SQL basics, data wrangling, SQL analysis, AB testing, distributed computing using Apache Spark, Delta Lake and more. These topics will prepare you to apply SQL creatively to analyze and explore data; demonstrate efficiency in writing queries; create data analysis datasets; conduct feature engineering, use SQL with other data analysis and machine learning toolsets; and use SQL with unstructured data sets.

Frequently Asked Questions

When will I have access to the lectures and assignments?
Access to lectures and assignments depends on your type of enrollment. If you take a course in audit mode, you will be able to see most course materials for free. To access graded assignments and to earn a Certificate, you will need to purchase the Certificate experience, during or after your audit. If you don't see the audit option:
The course may not offer an audit option. You can try a Free Trial instead, or apply for Financial Aid.
The course may offer 'Full Course, No Certificate' instead. This option lets you see all course materials, submit required assessments, and get a final grade. This also means that you will not be able to purchase a Certificate experience.
What will I get if I subscribe to this Specialization?
When you enroll in the course, you get access to all of the courses in the Specialization, and you earn a certificate when you complete the work. Your electronic Certificate will be added to your Accomplishments page - from there, you can print your Certificate or add it to your LinkedIn profile. If you only want to read and view the course content, you can audit the course for free.
What is the refund policy?
If you subscribed, you get a 7-day free trial during which you can cancel at no penalty. After that, we don’t give refunds, but you can cancel your subscription at any time. See our full refund policy.
Is financial aid available?
Yes. In select learning programs, you can apply for financial aid or a scholarship if you can’t afford the enrollment fee. If fin aid or scholarship is available for your learning program selection, you’ll find a link to apply on the description page.

More questions? Visit the Learner Help Center.

Build employee skills, drive business results

Discover Coursera for Business

Distributed Computing with Spark SQL

About this Course

What you will learn

Skills you will gain