Information and Entropy

View Course Materials

Buying via our links supports this free directory · Affiliate disclosure.
Course facts
  • Payment: Free, no certificate
  • Credits: Non Credit
  • Learning style: Self-paced, start anytime
  • Resources: Open Textbooks; Problem Set Solutions; Problem Sets; Programming Assignments
MIT OpenCourseWare details
  • As taught in: Spring 2008
  • Department: Electrical Engineering and Computer Science, Mechanical Engineering
Created byMIT
Available onMIT OpenCourseWareMIT OpenCourseWare

Course Details
Learning Style

Start Anytime

Level

All Levels

Study Mode

100% Online

Outcomes
Academic Credits

Non Credit

Price & Terms
Pricing Model

Free Content (No Certificate)

Technical
Content Medium

Text Based

Specification
Access

Free Access

Academic Level

Undergraduate

SKU: information-entropy Category: Brand:

Information and entropy: bits, codes, compression, noise, probability, communications, inference, maximum entropy and quantum information.

At a glance
LevelAll Levels
AccessFree access
FormatText Based
As taught inSpring 2008
DepartmentElectrical Engineering and Computer Science, Mechanical Engineering
Add your review
Add to wishlistAdded to wishlistRemoved from wishlist 0
Add to compare

What are the ultimate limits to communication and computation, and what does information have to do with physics? This course for first-year students explores the physical nature of information and its processing, unit by unit: bits and codes, compression, noise and error correction, probability, communications, processes and inference, then maximum entropy, physical systems, energy, temperature and quantum information. It is taught from a full set of course notes by Professor Penfield.

What You Will Achieve

  • Represent information in bits and design codes to store and send it
  • Apply compression and error-correcting codes to data in the presence of noise
  • Use probability and inference to reason about uncertain information
  • Connect the information-theoretic idea of entropy to energy and temperature in physical systems


Program Overview

Bits, Codes and Compression

Representing and packing information.

  • Bits and codes
  • Compression
  • Noise and errors

Probability, Communication and Inference

Information under uncertainty.

  • Probability
  • Communications
  • Processes
  • Inference

Entropy and Physics

Where information meets thermodynamics and quantum mechanics.

  • Maximum entropy
  • Physical systems, energy and temperature
  • Quantum information

Course Completion


Job Outlook

Information theory underpins communications engineering, data compression, computer science, statistical physics and quantum computing.


Before you start: Physics I: Classical Mechanics (Offered in various forms at MIT as 8.01SC Classical Mechanics, 8.012 Physics I: Classical Mechanics, 8.01L Physics I: Classical Mechanics, 8.01T, or advanced standing credit)

Our take

Good for: Curious beginners in science or engineering who want to see how information, entropy and physics fit together.

Keep in mind: The problem sets on the site are representative ones from 2003, some with solutions, and there are no lecture videos; the course notes carry the material.

Bottom line: An unusual, illuminating first look at information as a physical quantity, built on excellent course notes.

Frequently Asked Questions

What are the course notes?

A complete set of notes written by Professor Penfield over several years of teaching the subject, linked unit by unit and published as an open textbook.

Are there problem sets to practice with?

Yes. Problem sets from 2003 are republished with each unit as representative homework, some of them with solutions, along with programming assignments.

How does the course link information to physics?

After coding, compression and inference, the later units apply the principle of maximum entropy to physical systems, energy and temperature, and end with quantum information.

User Reviews

0.0 out of 5
★★★★★
0
★★★★★
0
★★★★★
0
★★★★★
0
★★★★★
0

No reviews yet. Be the first to review this course.

There are no reviews yet.

Write your review

Reviews can only be posted by registered learners. Log in, then you will return here to post.

×

    Report this page
    LiveTalent
    Logo
    Compare items
    • Total (0)
    Compare
    0