Introduction to Data Quality with Great Expectations
| Course Details | ||||||||
|---|---|---|---|---|---|---|---|---|
|
||||||||
| Languages | ||||||||
|
||||||||
| Outcomes | ||||||||
|
||||||||
| Price & Terms | ||||||||
|
||||||||
| Technical | ||||||||
|
||||||||
| Specification | ||||||||
|
||||||||
Learn to write, validate and deploy data quality checks in Python with Great Expectations, using suites, Checkpoints and conditional rules.
Great Expectations (GX) monitors data quality in Python through Expectations, assertions about data that the library verifies, so you can use a dataset knowing it meets your quality standards. You link the library to a pandas dataframe, test its shape, schema and values, group checks into suites, and run them through Checkpoints when new data arrives. Numeric, string and conditional rules then give finer control over individual columns.
What You Will Achieve
- Start a GX session and load a pandas dataframe as a batch ready for testing
- Write checks on column count, column names and value ranges, then read the validation results
- Bundle related checks into a suite and run it against fresh data through a Checkpoint
- Write numeric and string Expectations, and conditional ones that test only the rows meeting a condition
Program Overview
Connecting GX to Data
Sets up a Great Expectations session and brings a pandas dataframe into the library through its chain of data objects.
- Why GX is used to monitor data quality
- Starting a session with a Data Context
- Data Sources, Data Assets and Batch Definitions
- Reading and inspecting a batch
Validating Expectations and Managing GX Components
Builds shape and schema checks, validates them alone or as suites, and deploys and maintains them as data changes.
- Column count, column name and range checks
- Expectation Suites and Validation Definitions
- Deploying validations with Checkpoints
- Updating, copying, deleting, adding and listing components
Column, Numeric, String and Conditional Expectations
Surveys the main kinds of Expectations and shows how to apply a rule to only part of a dataframe.
- Row-level and aggregate-level column checks
- Numeric and string Expectations, including parseability
- Conditional Expectations and inverting a condition
Certificate Awarded (Paid)
Job Outlook
Automated data quality checks belong in data engineering and data science work, wherever Python pipelines deliver datasets that analysts and engineers need to rely on.
Good for: Analysts and data engineers working in Python who want repeatable, automated checks on the datasets their reports and pipelines depend on.
Keep in mind: Everything is taught through the GX Core library and its own terms, so the checks you learn are tied to that tool and to pandas dataframes.
Bottom line: Its strength is practicing the whole GX Core workflow: connecting data, building suites, deploying Checkpoints and writing conditional rules.
Frequently Asked Questions
What is an Expectation in Great Expectations?
An Expectation is an assertion you want to verify about your data, such as how many columns a table has or which range a column's values fall in. The course has you create them and validate them one at a time or grouped in a suite.
Which part of the Great Expectations library does the course use?
The lessons work with GX Core, the Python library of Great Expectations, and rely on pandas Data Sources and dataframes throughout, so every check you write is applied to a dataframe loaded into GX.
How is the practice organized in this course?
Short exercises follow each video and ask you to apply what was just shown: creating Expectations, validating data and managing GX components. The exercises work with real-world datasets.
Can an Expectation check only some rows of a dataframe?
Yes. The final chapter covers conditional Expectations, which test one column only on rows where other columns meet a condition, and shows how to invert that condition when you need the opposite subset.
User Reviews
Write your review
You must be logged in to post a review.
Reviews can only be posted by registered learners. Log in or create a free account, then you will return here to post.
There are no reviews yet.