Unit Testing for Data Science in Python

Course Description

Every data science project needs unit testing. It comes with huge benefits – saving a lot of development and maintenance time, improving documentation, increasing end-user trust and reducing downtime of productive systems. As a result, unit testing has become a must-have skill in the industry, used by almost every company. This course teaches unit testing in Python using the most popular testing framework pytest. By the end of this course, you will have written a complete test suite for a data science project. In the process, you will learn to write unit tests for data preprocessors, models and visualizations, interpret test results and fix any buggy code. You will also learn advanced concepts like TDD, test organization, fixtures and mocking so that you can test your own data science projects properly.

What You’ll Learn

Unit testing basics

In this chapter, you will get introduced to the pytest package and use it to write simple unit tests. You’ll run the tests, interpret the test result reports and fix bugs. Throughout the chapter, we will use examples exclusively from the data preprocessing module of a linear regression project, making sure you learn unit testing in the context of data science.

Test Organization and Execution

In any data science project, you quickly reach a point when it becomes impossible to organize and manage unit tests. In this chapter, we will learn about how to structure your test suite well, how to effortlessly execute any subset of tests and how to mark problematic tests so that your test suite always stays green. The last lesson will even enable you to add the trust-inspiring build status and code coverage badges to your own project. Complete this chapter and become a unit testing wizard!

Intermediate unit testing

In this chapter, you will write more advanced unit tests. Starting from testing complicated data types like NumPy arrays to testing exception handling, you’ll do it all. Once you have mastered the science of testing, we will also focus on the arts. For example, we will learn how to find the balance between writing too many tests and too few tests. In the last lesson, you will get introduced to a radically new programming methodology called Test Driven Development (TDD) and put it to practice. This might actually change the way you code forever!

Testing Models, Plots and Much More

In this chapter, You will pick up advanced unit testing skills like setup, teardown and mocking. You will also learn how to write sanity tests for your data science models and how to test matplotlib plots. By the end of this chapter, you will be ready to test real world data science projects!