SyntaxPilot
Courses/Data Analytics with Python/Lesson 3
Video coming soonThis lesson’s video is being made

Start with the notes below; the video appears here as soon as it is on YouTube.

Data Analytics with Python · Lesson 3 of 90

Setting up: Jupyter, Colab and Python environments

PythoninterpreterJupyter NotebookJupyterLabGoogle Colabcellcode cellmarkdown cellkernelrestart and run allAnacondapippackagelibrary

These are the notes for Data Analytics with Python, episode 3. You will learn why analysts work in notebooks instead of plain scripts, what cells, code cells, markdown cells and the kernel are, the hidden-state trap and Restart and Run All, the four places notebooks run (Jupyter Notebook, JupyterLab, Google Colab, VS Code), what import, modules, libraries and packages are, how pip installs packages from PyPI, and how virtual environments (venv, conda) and requirements.txt keep projects from breaking each other. Every term is defined the first time it appears and collected in the Glossary. Every program and terminal session below was run for real (Python 3.13); outputs are unedited.

Two words first: Python and the interpreter

  • Python is a programming language: a way to write instructions for a computer in something close to English.
  • The interpreter is the program that reads your Python and runs it, line by line, like a live translator at a meeting.
  • A script is a .py file that the interpreter runs from top to bottom.

The problem: every tweak reloads everything

An analyst loads a year of sales (slow), then prints a chart title. time.sleep(5) stands in for a slow file read: it just waits 5 seconds.

import time
print("Loading a year of sales...")
time.sleep(5)   # pretend: reading a huge file
print("Loaded.")
print("Chart: sunny vs rainy")
Loading a year of sales...
Loaded.
Chart: sunny vs rainy

Her manager asks to capitalise the title. She changes one line, runs again, and waits another 5 seconds for the whole load. With real data the wait is often minutes, on every tweak.

Why: a script always runs from the top, and when it ends, Python forgets every variable. Fix: keep Python running and run one small piece at a time. That's a notebook.

Notebooks: cells and the kernel

Restaurant Notebook Meaning
Ordering one dish at a time Cell One small piece of the notebook you run on its own
— Code cell A cell that holds Python code; its output appears right below it
The waiter who remembers your table Kernel A Python process kept running behind the notebook; it runs your cells and remembers every variable
The menu's descriptions Markdown cell A cell of formatted text (headings, notes), not code

Markdown is a simple way to format text: # Heading, **bold**, - bullet. Mixing markdown cells and code cells makes a notebook read like a report.

The analyst's work as two cells:

# Cell [1]  (run once: 5 seconds)
import time
print("Loading a year of sales...")
time.sleep(5)
sales = 1263
print("Loaded.")
# Cell [2]  (re-run as often as you like: instant)
print("Chart: Sunny vs Rainy days, total", sales)
[1]  Loading a year of sales...
     Loaded.
[2]  Chart: Sunny vs Rainy days, total 1263

Cell 2 re-runs instantly because the kernel still remembers sales. Notebook files end in .ipynb (short for IPython notebook); inside, they are a JSON file holding the cells and their outputs.

The trap: hidden state

  1. You run cell 1 (total = 1263), then cell 2 (print(total * 2)). Works.
  2. You delete cell 1, or move cell 2 above it. It still works, because the kernel remembers total. Memory you can't see is called hidden state.
  3. A teammate opens the notebook; their fresh kernel runs top to bottom, and it crashes.

The same code as a script (which, like a fresh kernel, always runs top to bottom):

# Cell [2] was written first in this file...
print(total * 2)
# ...and Cell [1] below it
total = 1263
Traceback (most recent call last):
  ...
NameError: name 'total' is not defined

Fix: Restart and Run All. Restart wipes the kernel's memory; Run All runs every cell in order, exactly as your teammate will. Do it before you share any notebook: if it survives Restart and Run All, it works for everyone.

Where notebooks run

Tool What it is Install? Notes
Jupyter Notebook The classic notebook app; runs on your computer, opens in your browser Yes Name from Julia, Python and R. Project Jupyter was spun off from IPython in 2014
JupyterLab Jupyter's newer interface: files, notebooks and terminals side by side Yes Same .ipynb files
Google Colab A hosted Jupyter notebook service (runs on Google's computers) No Free to use; notebooks saved in Google Drive (or loaded from GitHub); resources aren't guaranteed, and idle machines are deleted (variables are lost)
VS Code A free code editor Yes Opens .ipynb files through the Jupyter extension; has Restart, Run All, a kernel picker and a Variables panel

Hosted means the software runs on someone else's computers and you use it through the browser.

import, modules, libraries and packages

Python on its own can't read Excel files or draw charts. You bring in ready-made code with the import statement.

import math
print(math.sqrt(144))
import pandas as pd
print(pd.__version__)
import polars
12.0
3.0.5
Traceback (most recent call last):
  ...
ModuleNotFoundError: No module named 'polars'
  • import math loads Python's built-in maths module; math.sqrt(144) is 12.0.
  • import pandas as pd loads pandas (the main data library of this course) under the short nickname pd. pd.__version__ shows which version is installed (yours may differ).
  • polars (another fast data library) isn't installed on this machine, so Python raises ModuleNotFoundError.
Kitchen Python Meaning
One recipe card Module One .py file of ready-made code (e.g. math)
A whole cookbook Library A collection of ready-made code for one kind of job (e.g. pandas for tables)
The cookbook, packed for sale Package A library bundled so it can be installed. In everyday talk, library and package mean the same thing
The supermarket PyPI The Python Package Index: the online store of free packages
Home delivery pip The installer: "the PyPA recommended tool for installing Python packages", downloads from PyPI

The fix: pip install

Run this in a terminal (the text window where you type commands). In a notebook cell, write %pip install polars.

$ pip install polars
  ... (download progress lines skipped) ...
Installing collected packages: polars-runtime-32, polars

Successfully installed polars-2.1.0 polars-runtime-32-2.1.0
$ python -c "import polars; print('polars works')"
polars works

pip also installed polars-runtime-32, a package polars needs. Packages that a package needs are its dependencies.

The next problem: one Python, many projects

Last year's report was built with pandas 2; this year's dashboard needs pandas 3. Versions really do differ: this machine has pandas 3.0.5, while a fresh install a minute later got 3.1.0. If every project shares one Python, upgrading for one project can break another.

Fix: a virtual environment (like a tiffin with separate boxes): a separate, private set of packages for one project, built on top of your normal Python. The Python docs describe it as "isolated from the packages in the base environment, so that only those explicitly installed in the virtual environment are available".

venv: create, activate, install (a real session)

$ mkdir shop && cd shop
$ python3 -m venv .venv
$ source .venv/bin/activate
(.venv) $ python -c "import pandas"
ModuleNotFoundError: No module named 'pandas'
(.venv) $ pip install -q pandas
(.venv) $ python -c "import pandas as pd; print(pd.__version__)"
3.1.0
Step macOS / Linux Windows (cmd) Windows (PowerShell)
Create python3 -m venv .venv python -m venv .venv python -m venv .venv
Activate source .venv/bin/activate .venv\Scripts\activate.bat .venv\Scripts\Activate.ps1
Leave deactivate deactivate deactivate
  • venv is the tool built into Python for virtual environments; .venv is just the folder name we chose.
  • After activating, the prompt starts with (.venv), so you know which environment you're in.
  • The new environment starts empty: import pandas failed even though this computer has pandas. That's the isolation working.

requirements.txt: the label on the lid

(.venv) $ pip freeze > requirements.txt
(.venv) $ cat requirements.txt
numpy==2.5.4
pandas==3.1.0
python-dateutil==3.0.0
six==1.17.0
tzdata==2026.5
# a teammate rebuilds the same setup with:
$ pip install -r requirements.txt

pip freeze lists every installed package with its exact version. A requirements file stores that list so anyone can rebuild the same environment with pip install -r requirements.txt. Environments are disposable: never put your own code inside one, don't commit it to Git, and recreate it from the requirements file when needed.

Anaconda and conda

What it is
Anaconda Distribution One installer with Python, conda, Anaconda Navigator and 600+ data science packages
conda A package manager and environment manager (conda create, conda install): pip's and venv's jobs together
Anaconda Navigator A desktop app to create environments and launch Jupyter with clicks
Trade-off A big download that uses gigabytes of disk; venv + pip is lighter

Pick one approach per project and stick with it.

Which setup should you use?

Situation Use
Just starting, or a slow laptop Google Colab
Your own laptop (the setup used at jobs) Python + VS Code + Jupyter extension, one venv per project
Quick snippet The SyntaxPilot online compiler

Common mistakes (with the real error messages)

Mistake Real message Why Fix
Using a variable from a deleted or later cell NameError: name 'total' is not defined Hidden state Restart and Run All before sharing
Importing a package that isn't installed ModuleNotFoundError: No module named 'polars' Not installed in this Python pip install polars (%pip install in a notebook)
pip install pandas worked, but import pandas fails ModuleNotFoundError: No module named 'pandas' pip installed into a different environment Activate the right venv / pick the right kernel
Putting project files inside .venv or committing it (no error: a huge, broken repo) Environments are disposable Commit requirements.txt instead

Mini project: a reproducible analysis folder

On your laptop (or in a Colab terminal):

$ mkdir chai_analysis && cd chai_analysis
$ python3 -m venv .venv
$ source .venv/bin/activate
(.venv) $ pip install pandas
(.venv) $ pip freeze > requirements.txt

Then create analysis.ipynb with: a markdown cell # Do rainy days hurt chai sales?, a code cell with episode 1's week list, a code cell that cleans it, and a code cell that prints the averages. Finish with Restart and Run All: every cell should run in order with no errors.

Practice

1. Which is which? (a) runs your cells and remembers variables, (b) holds a heading and notes, (c) the online store of packages, (d) the installer.

Solution

(a) kernel, (b) markdown cell, (c) PyPI, (d) pip.

2. You run pip install seaborn in a terminal, but your notebook still says ModuleNotFoundError: No module named 'seaborn'. Give two likely causes.

Solution

The terminal and the notebook use different environments (the venv wasn't activated, or the notebook's kernel is a different Python); or the kernel started before the install and needs a restart. Fix: %pip install seaborn inside the notebook (installs into the kernel's own environment), then restart the kernel.

3. Make a NameError on purpose in a notebook: cell A x = 10, cell B print(x * 2). Run both, delete cell A, run cell B again (it works). Now Restart and Run All. What happens and why?

Solution

After the restart, cell B raises NameError: name 'x' is not defined: x only existed in the old kernel's memory (hidden state).

4. Your teammate needs your exact package versions. What do you send them, and what command do they run?

Solution

Send requirements.txt (made with pip freeze > requirements.txt). They create and activate their own venv and run pip install -r requirements.txt.

Quick quiz

Your notebook worked all day. You click Restart and Run All, and cell [4] fails: NameError: name 'clean' is not defined. Most likely cause?

A. pandas isn't installed B. A cell that creates clean was deleted or sits below cell [4] C. Colab is down D. The kernel is too slow

Answer

B. clean only existed in the old kernel's memory. Restart and Run All exposed the hidden state before your teammate did.

Interview questions

  1. Why do analysts use notebooks? Run code in small cells, keep data in memory between steps, and mix code, output, charts and explanations in one document.
  2. What is a kernel? The running language process behind a notebook that executes cells and holds variables.
  3. What is hidden state, and how do you avoid it? Variables in kernel memory that the visible cells no longer create; avoid it with Restart and Run All before sharing.
  4. Library vs module vs package? Module = one file of code; library = a collection for one job; package = how it's bundled for installing (often used interchangeably with library).
  5. Why use virtual environments? To give each project its own package versions so projects can't break each other, and to make setups reproducible with requirements.txt.
  6. venv vs conda? venv (with pip) is built into Python and light; conda manages both packages and environments and comes with Anaconda.

Glossary

Term Meaning Example
Python A programming language close to English print("hi")
Interpreter The program that reads and runs Python line by line python3 main.py
Script A .py file run top to bottom analysis.py
Notebook A document of cells you run one at a time analysis.ipynb
Jupyter Notebook The classic notebook app (Julia, Python, R) Opens in your browser
JupyterLab Jupyter's newer interface with files and terminals Side-by-side panels
Google Colab A free hosted Jupyter notebook service colab.research.google.com
Hosted Runs on someone else's computers, used via the browser Colab
Cell One piece of a notebook you run on its own Cell [2]
Code cell A cell holding code, with its output below print(sales)
Markdown cell A cell of formatted text # Question
Markdown A simple text formatting syntax **bold**
Kernel The running Python behind a notebook that remembers variables Restarting it clears memory
Hidden state Variables in memory that no visible cell creates A deleted cell's total
Restart and Run All Wipe the kernel, then run every cell in order Do it before sharing
.ipynb The notebook file format (JSON inside) analysis.ipynb
VS Code A free code editor; opens notebooks via the Jupyter extension —
import statement Loads a module or library into your program import pandas as pd
Module One .py file of ready-made code math
Library A collection of ready-made code for one job pandas
Package A library bundled for installing polars
Dependency A package another package needs numpy for pandas
PyPI The Python Package Index, the online store of packages pypi.org
pip The tool that installs packages from PyPI pip install polars
Terminal A text window for typing commands $ pip --version
ModuleNotFoundError Python's error when an imported module isn't installed No module named 'polars'
NameError Python's error for a name that doesn't exist yet name 'total' is not defined
Virtual environment A private set of packages for one project .venv
venv Python's built-in tool for virtual environments python3 -m venv .venv
Activate Switch your terminal into an environment source .venv/bin/activate
requirements.txt A list of packages with exact versions pandas==3.1.0
pip freeze Prints installed packages with versions pip freeze > requirements.txt
Anaconda Distribution One installer: Python, conda, Navigator, 600+ packages —
conda A package and environment manager conda create -n shop pandas
Anaconda Navigator A desktop app for environments and Jupyter —

Summary

  • A script reruns everything from the top; a notebook runs cells one at a time while the kernel remembers variables.
  • Hidden state is the notebook trap; Restart and Run All is the fix.
  • Notebooks run in Jupyter Notebook, JupyterLab, Google Colab and VS Code.
  • import loads modules and libraries; pip installs packages from PyPI; missing ones raise ModuleNotFoundError.
  • Virtual environments (venv or conda) give each project its own packages; requirements.txt makes the setup reproducible.
  • Next episode: Python for data, part 1: variables, types and operators.