A/B Testing in Python

Start Free Preview

Buying via our links supports this free directory · Affiliate disclosure.
Course facts
  • Payment: Free Preview (Paid Upgrade)
  • Learning style: Self-paced, start anytime
  • Format: Text & Interactive
  • Video access: Streaming Only
  • Before you start: Hypothesis Testing in Python
DataCamp details
  • Chapters: 4
Created byDataCamp
Available onDataCamp

Course Details
Learning Style

Start Anytime

Level

Intermediate

Study Mode

100% Online

Duration

Short (< 10 Hours)

Languages
Audio Language

English

Outcomes
Certificate

Certificate of Completion

Price & Terms
Pricing Model

Free Preview (Paid Upgrade)

Technical
Content Medium

Text & Interactive

Video Access

Streaming Only

Specification
Access

Subscription

SKU: a-b-testing-python Category: Tags: , Brand:

Learn to plan A/B tests in Python, size the sample, run validity checks and analyze proportions, means and ratio metrics to support a decision.

At a glance
Duration4 hrs
LevelIntermediate
CertificateCertificate of Completion
LanguageEnglish
AccessSubscription
Add your review
Add to wishlistAdded to wishlistRemoved from wishlist 0
Add to compare

A/B testing compares two variants of a web page, ad or campaign on randomly split groups of users to see which one performs better. A reliable answer depends on choosing the right metric, collecting a large enough sample, and checking the data before any p-value is trusted. You work through each of these steps in Python, then analyze differences in proportions, means and ratio metrics so the result can support a business decision.

What You Will Achieve

  • Turn a business question into a testable hypothesis with a fitting metric
  • Estimate how many users and how much time a test needs to detect a meaningful effect
  • Run internal and external validity checks on test data before reporting a result
  • Pick a parametric or non-parametric test that suits the data and interpret its p-value and confidence interval


Program Overview

A/B Testing Foundations and Metrics

Why companies run experiments, when an A/B test is the wrong tool, and how metrics are designed and estimated.

  • Randomization and the steps of a test
  • Correlation versus causation
  • Means and proportions as metrics
  • An ad impressions example

Experiment Design, Power and Sample Size

Planning a test before it runs, with strong hypotheses, error rates and the sample needed for a conclusive result.

  • Hypothesis formulation and the central limit theorem
  • P-values, error rates and minimum detectable effects
  • Power curves and sample size for means and proportions
  • Multiple comparisons and corrected p-values

Sanity Checks, Results Analysis and Decisions

Cleaning and exploring test data, checking validity, and analyzing differences in proportions, means and ratio metrics before making a decision.

  • Validity checks for SRM, novelty effects and Simpson’s paradox
  • Difference in proportions tests and confidence intervals
  • T-tests, Mann-Whitney U and chi-square tests
  • The delta method, best practices and the day-of-the-week effect

Certificate Awarded (Paid)

Complete the course to earn it.


Job Outlook

A/B testing skills are used wherever teams compare versions of a web page, ad or marketing campaign, including e-commerce sites, search engines, social networks and marketing teams.


Our take

Good for: Analysts, data scientists and online marketers who need to plan experiments carefully and defend the result to the people making the decision.

Keep in mind: Although rated intermediate, the course is aimed at advanced learners with strong Python experience, so expect the lessons to assume that confidence.

Bottom line: It walks through a full A/B test workflow in Python, giving the sanity checks and the choice of test as much weight as the p-value.

Frequently Asked Questions

Which statistical tests does the course use?

The analysis lessons use tests for a difference in proportions, t-tests for a difference in means including pairwise t-tests, the Mann-Whitney U test as a non-parametric option, and the chi-square test for independence. Ratio metrics are handled with the delta method.

What sanity checks are taught before results are analyzed?

Internal validity checks cover SRM and whether distributions are balanced between groups. External validity checks cover detecting novelty effects, and a worked example shows Simpson's paradox. A best-practices lesson also looks at the day-of-the-week effect.

How does the course handle multiple comparisons?

The experiment design chapter covers multiple comparisons tests, how to judge whether a correction is needed, and how to compute corrected p-values. The final chapter adds pairwise t-tests for comparing means across groups.

User Reviews

0.0 out of 5
★★★★★
0
★★★★★
0
★★★★★
0
★★★★★
0
★★★★★
0

No reviews yet. Be the first to review this course.

There are no reviews yet.

Write your review

Reviews can only be posted by registered learners. Log in, then you will return here to post.

×

    Report this page
    LiveTalent
    Logo
    Compare items
    • Total (0)
    Compare
    0