Building Recommendation Engines with PySpark

Start Free Preview

Buying via our links supports this free directory · Affiliate disclosure.
Course facts
  • Payment: Free Preview (Paid Upgrade)
  • Learning style: Self-paced, start anytime
  • Format: Text & Interactive
  • Video access: Streaming Only
  • Before you start: Supervised Learning with scikit-learn; Introduction to PySpark
DataCamp details
  • Chapters: 4
Created byDataCamp
Available onDataCamp

Course Details
Learning Style

Start Anytime

Level

Advanced

Study Mode

100% Online

Duration

Short (< 10 Hours)

Languages
Audio Language

English

Outcomes
Certificate

Certificate of Completion

Price & Terms
Pricing Model

Free Preview (Paid Upgrade)

Technical
Content Medium

Text & Interactive

Video Access

Streaming Only

Specification
Access

Subscription

SKU: building-recommendation-engines-pyspark Category: Tags: , , , Brand:

Learn to build, tune and evaluate ALS recommendation engines in PySpark on explicit movie ratings and ratings inferred from customer behavior data.

At a glance
Duration4 hrs
LevelAdvanced
CertificateCertificate of Completion
LanguageEnglish
Taught byJamen Long
AccessSubscription
Add your review
Add to wishlistAdded to wishlistRemoved from wishlist 0
Add to compare

Recommendation engines here are built with Alternating Least Squares (ALS) in PySpark, a matrix factorization method that predicts the ratings missing from a large user-item matrix by uncovering latent features. You work through the intuition and the math behind ALS, then write the code to train, test and tune models on movie ratings. Because real customer data seldom holds clean ratings, you also infer ratings from customer behavior and build implicit-feedback models on them.

What You Will Achieve

  • Tell whether a recommendation problem suits collaborative filtering and whether its data gives explicit or implicit ratings
  • Shape a Spark DataFrame for ALS, with distinct users and integer IDs assigned to each item
  • Fit an ALS model on a train/test split, tune its hyperparameters with cross-validation and read the resulting RMSE
  • Turn customer behavior data into inferred ratings and build ALS recommenders from them


Program Overview

Recommendation Engine Types and Latent Features

Shows what recommendation engines can do and the distinctions that decide how one is built.

  • Collaborative vs content-based filtering
  • Implicit and explicit ratings data
  • Other uses for recommendation engines
  • Hidden latent features in customer datasets

How ALS Works and Data Preparation for Spark

Reviews the matrix math behind ALS, then readies data and builds a first model with an evaluator.

  • Matrix multiplication and matrix factorization
  • Non-negative matrix factorization
  • How ALS alternates to estimate predictions
  • ALS parameters, hyperparameters and an RMSE evaluator

Building ALS Models on Movies and Songs

Applies ALS to the MovieLens data, then handles the case with no customer ratings using the Million Songs Dataset.

  • MovieLens sparsity and summary statistics
  • Cross-validation, best model parameters and sanity checks on recommendations
  • Implicit ratings, added zeros and their evaluation
  • Binary implicit ratings models

Certificate Awarded (Paid)


Job Outlook

These skills apply wherever customer data feeds product suggestions, such as the movie and music catalogs used in the exercises, and in data science and data engineering work that runs on Spark.


Our take

Good for: Learners already working with Spark who want to build collaborative-filtering recommenders and judge them with RMSE and cross-validation.

Keep in mind: Every model you build is an ALS collaborative-filtering model; content-based engines appear only in the opening comparison of engine types.

Bottom line: A focused run through ALS in PySpark that pairs the matrix math with cross-validated models on explicit movie ratings and implicit song data.

Frequently Asked Questions

What is matrix factorization and how does ALS use it?

Matrix factorization splits a large user-item matrix into smaller matrices whose product approximates it, exposing latent features that help predict missing ratings. ALS builds on it, and the lessons show how it alternates to generate predictions and how RMSE changes as it does.

Which datasets do the exercises use?

The MovieLens dataset carries the explicit-ratings work, including a full cross-validated ALS model. The Million Songs Dataset and a second version of MovieLens are used for the chapter on implicit and binary ratings.

How are ratings inferred when customers never rate anything?

You start from customer behavior data, compute summary and grouped statistics, add zeros to the data, and then fit implicit ALS models with their own hyperparameters. A binary version of the ratings is also modeled and its recommendations reviewed.

How does the course judge whether recommendations are good?

Explicit models are scored with an RMSE evaluator, and a lesson covers what the RMSE value means. You also check whether the recommendations make sense, and implicit models get their own evaluation approach.

User Reviews

0.0 out of 5
★★★★★
0
★★★★★
0
★★★★★
0
★★★★★
0
★★★★★
0

No reviews yet. Be the first to review this course.

There are no reviews yet.

Write your review

Reviews can only be posted by registered learners. Log in, then you will return here to post.

×

    Report this page
    Building Recommendation Engines with PySpark
    Building Recommendation Engines with PySpark
    LiveTalent
    Logo
    Compare items
    • Total (0)
    Compare
    0