Monitoring Machine Learning in Python

Start Free Preview

Buying via our links supports this free directory · Affiliate disclosure.
Course facts
  • Payment: Free Preview (Paid Upgrade)
  • Learning style: Self-paced, start anytime
  • Format: Text & Interactive
  • Video access: Streaming Only
  • Before you start: Monitoring Machine Learning Concepts
DataCamp details
  • Chapters: 3
Created byDataCamp
Available onDataCamp

Course Details
Learning Style

Start Anytime

Level

Advanced

Study Mode

100% Online

Duration

Short (< 10 Hours)

Languages
Audio Language

English

Outcomes
Certificate

Certificate of Completion

Price & Terms
Pricing Model

Free Preview (Paid Upgrade)

Technical
Content Medium

Text & Interactive

Video Access

Streaming Only

Specification
Access

Subscription

Build a model monitoring workflow in Python with NannyML: estimate performance without labels, measure business value and trace drift to its cause.

At a glance
Duration3 hrs
LevelAdvanced
CertificateCertificate of Completion
LanguageEnglish
AccessSubscription
Add your review
Add to wishlistAdded to wishlistRemoved from wishlist 0
Add to compare

A machine learning model that scored well at launch can quietly lose accuracy once production data shifts, and the true labels needed to measure it often arrive late or never. Working in Python with the NannyML library, you estimate how a model performs before ground truth exists, check those estimates against realized results later, and trace a drop back to drifting features or data quality problems so you can decide what, if anything, needs fixing.

What You Will Achieve

  • Split raw production data into reference and analysis sets that NannyML can monitor
  • Estimate a deployed model’s performance without labels using the CBPE and DLE workflow
  • Compare estimated and realized results and express a performance drop as lost business value
  • Rank drifting features with univariate and multivariate methods to find the root cause of model issues


Program Overview

Data Preparation and Performance Estimation with NannyML

Introduces NannyML, then readies the New York taxi tip data and estimates how the tip prediction model performs in production.

  • Key features of NannyML
  • Reference and analysis periods
  • Specifying the algorithm and problem type
  • CBPE and DLE workflow

Realized Performance and Business Value

Covers realized performance calculators for when ground truth arrives, ways of handling results, and the business value of a hotel booking model.

  • Estimated versus realized performance
  • Filtering, plotting and chunking results
  • Custom thresholds
  • Business value and drops in monetary value

Drift Detection, Data Quality and Issue Resolution

Traces the hotel booking model’s degradation to its underlying cause and closes by putting a full monitoring workflow together.

  • Multivariate and univariate drift detection
  • Ranking and visualizing drifting features
  • Data quality and summary statistics checks
  • Choosing a resolution, including doing nothing

Certificate Awarded (Paid)


Job Outlook

These skills are used wherever machine learning models run in production and need to stay accurate over time, such as on MLOps, data science and machine learning engineering teams.


Our take

Good for: Python practitioners who already deploy models and want a hands-on way to watch them in production when labels arrive late or not at all.

Keep in mind: The monitoring system is built with one package, NannyML, so the workflow is learned through its functions on two practice datasets.

Bottom line: A focused NannyML course that follows two datasets through estimating performance without labels, valuing the loss and tracing it to drifting features.

Frequently Asked Questions

Which models does the course monitor?

The first chapter works with New York taxi rides and a model that predicts tip amounts. The later chapters switch to a hotel booking dataset for realized performance, business value and root cause analysis.

How does the course handle missing ground truth labels?

The opening chapter estimates model performance with NannyML before labels exist, using the CBPE and DLE workflow. The second chapter turns to realized performance calculators once ground truth is available and compares the two, including cases where the estimate is off.

Does the course end with a complete monitoring workflow?

Yes. The closing lessons cover issue resolution, including whether doing nothing is the right call, and then implementing a monitoring workflow that brings the course's steps together.

User Reviews

0.0 out of 5
★★★★★
0
★★★★★
0
★★★★★
0
★★★★★
0
★★★★★
0

No reviews yet. Be the first to review this course.

There are no reviews yet.

Write your review

Reviews can only be posted by registered learners. Log in, then you will return here to post.

×

    Report this page
    Monitoring Machine Learning in Python
    Monitoring Machine Learning in Python
    LiveTalent
    Logo
    Compare items
    • Total (0)
    Compare
    0