Web Scraping in Python

Start Free Preview

Buying via our links supports this free directory · Affiliate disclosure.
Course facts
  • Payment: Free Preview (Paid Upgrade)
  • Learning style: Self-paced, start anytime
  • Format: Text & Interactive
  • Video access: Streaming Only
  • Before you start: Intermediate Python
DataCamp details
  • Chapters: 4
Created byDataCamp
Available onDataCamp

Course Details
Learning Style

Start Anytime

Level

Intermediate

Study Mode

100% Online

Duration

Short (< 10 Hours)

Languages
Audio Language

English

Outcomes
Certificate

Certificate of Completion

Price & Terms
Pricing Model

Free Preview (Paid Upgrade)

Technical
Content Medium

Text & Interactive

Video Access

Streaming Only

Specification
Access

Subscription

SKU: web-scraping-python Category: Tags: , Brand:

Learn to parse HTML with XPath and CSS selectors in Python and build scrapy spiders that crawl websites and collect data automatically.

At a glance
Duration4 hrs
LevelIntermediate
CertificateCertificate of Completion
LanguageEnglish
AccessSubscription
Add your review
Add to wishlistAdded to wishlistRemoved from wishlist 0
Add to compare

Web scraping in Python is about getting useful information out of web pages and into your own data work. Pages keep that information inside HTML, so the first task is understanding how a document is arranged as a tree of elements, attributes and links. With that picture in place, you write XPath and CSS expressions that point at exactly the text you want, then turn them into automated crawlers with the scrapy library that follow links and gather details page after page.

What You Will Achieve

  • Read an HTML document as a tree and locate the elements, attributes and links that hold the data you need
  • Write and chain XPath and CSS expressions that extract text and href values from a page
  • Use scrapy Selector and Response objects to pull information out of a page’s source
  • Build a simple scrapy spider that starts from a set of URLs, follows links and parses each page automatically


Program Overview

HTML Structure and XPath Notation

Sets out why scraping is useful in data science, then shows how HTML documents are built and how XPath moves between their elements.

  • The value of web scraping for data science
  • HTML tree structure, tags and attributes
  • Class names and href links
  • Basic XPath notation

Selectors, CSS Locators and Response Objects

Goes deeper into XPath with scrapy selectors, adds CSS Locator syntax, combines the two by chaining, and introduces Response objects for work across several websites.

  • XPath attributes, text and chaining
  • Scrapy Selector objects
  • CSS Locators, wildcards and attribute selection
  • Response objects

Crawling the Web with Scrapy Spiders

Turns the selection techniques into scrapy spiders that visit multiple pages, follow links and scrape each page automatically, finishing with a capstone crawler.

  • Spider classes and start requests
  • Parsing pages and following links
  • Capstone crawler

Certificate Awarded (Paid)

Complete the course to earn it.


Job Outlook

Web scraping supports data science work wherever the information an analysis needs sits on websites instead of in a ready-made dataset, and the same HTML parsing skills are useful in web development tasks.


Our take

Good for: Python users who want to collect data from websites on their own and prefer learning one scraping library closely through hands-on coding exercises.

Keep in mind: All the scraping is done with scrapy; many of the techniques carry over to BeautifulSoup and Selenium, but the exercises do not use those libraries.

Bottom line: A focused path through HTML structure, XPath and CSS selection that ends with a working scrapy crawler you build yourself.

Frequently Asked Questions

What is XPath used for in web scraping?

XPath is a notation for moving through the elements of an HTML document and pointing at the ones you want to extract. The lessons introduce it early, use it with scrapy selectors, and later combine it with CSS Locators through chaining.

What is a Response object in scrapy?

A Response object behaves much like a Selector, so the same XPath and CSS expressions work on it, but it carries extra tools for taking your scraping across multiple websites.

Can I scrape websites of my own choosing afterwards?

The skills are meant to transfer: once you can read a page's HTML, write selectors and set up a spider that follows links, you can apply the same approach to other sites you want to collect data from. The capstone brings these steps together in one crawler.

User Reviews

0.0 out of 5
★★★★★
0
★★★★★
0
★★★★★
0
★★★★★
0
★★★★★
0

No reviews yet. Be the first to review this course.

There are no reviews yet.

Write your review

Reviews can only be posted by registered learners. Log in, then you will return here to post.

×

    Report this page
    LiveTalent
    Logo
    Compare items
    • Total (0)
    Compare
    0