These are the notes for Data Analytics with Python, episode 8. Python lists are great at holding things, but they can't do maths on a column of numbers, and looping over a million values is slow. Here we invent the fix: NumPy and its ndarray. You'll learn vectorisation and element-wise operations, the array's dtype, shape, ndim and size, quick builders (np.arange, np.linspace, np.zeros, np.ones, np.full), universal functions (ufuncs), and exactly why arrays beat lists. Every term is defined the first time it appears and collected in the Glossary. Every program below was run for real (Python 3.13, NumPy 2.x); outputs are unedited. Timings change from machine to machine and run to run.
The problem: lists can't do maths
Ramesh's menu is a list: chai Rs 12, samosa Rs 15, bun maska Rs 25. Milk got costlier, so he wants to raise every price by 10%, which means multiplying each one by 1.1. On paper you'd write prices * 1.1. Python disagrees.
prices = [12, 15, 25]
print(prices * 2)
print(prices + 5)
[12, 15, 25, 12, 15, 25]
Traceback (most recent call last):
File "main.py", line 3, in <module>
print(prices + 5)
~~~~~~~^~~
TypeError: can only concatenate list (not "int") to list
- For a list,
*means repeat the list, not multiply the numbers. - For a list,
+means concatenate: join two lists end to end. A number isn't a list, so it fails.
prices = [12, 15, 25]
print(prices * 1.1)
Traceback (most recent call last):
File "main.py", line 2, in <module>
print(prices * 1.1)
~~~~~~~^~~~~
TypeError: can't multiply sequence by non-int of type 'float'
A sequence is an ordered collection like a list or a string. You can repeat a list 2 times, but not 1.1 times. Lists were built to hold things, not to do maths.
The workaround from episode 6 is a comprehension, one price at a time:
prices = [12, 15, 25]
new = [p * 1.1 for p in prices]
print(new)
[13.200000000000001, 16.5, 27.500000000000004]
It works. The long tails like 13.200000000000001 are normal: computers store most decimals as a very close floating-point approximation. But Python walks the list one item at a time, which is slow for a million values.
NumPy and the ndarray
Real-life example: a list is a cloth jhola bag — anything goes in (eggs, keys, a phone), but to wash every egg you take them out one by one. An array is an egg tray — same-sized slots side by side, and you can wash the whole tray in one go.
- A library is a package of ready-made code you bring in with
import. - NumPy (Numerical Python) is the library for fast maths on arrays of numbers.
- Its main object is the ndarray ("N-dimensional array"): a grid of values that all have the same type.
np.array(...)turns a list (or a list of lists) into an ndarray.import numpy as np:asgives the library a short nickname;npis the name everyone uses.
import numpy as np
prices = np.array([12, 15, 25])
print(prices)
print(type(prices))
print(prices * 2)
print(prices + 5)
print(prices * 1.1)
[12 15 25]
<class 'numpy.ndarray'>
[24 30 50]
[17 20 30]
[13.2 16.5 27.5]
An array prints without commas. Now * 2 doubles every price and + 5 adds 5 to every price. The 10% rise is one line, with no loop. (The tiny decimal error is still stored; NumPy just prints fewer digits.)
Vectorisation and element-wise operations
- Vectorisation: writing one operation for the whole array, with no loop written by you. NumPy runs the loop internally.
- Element-wise: the operation is applied to each element (one value inside the array) on its own:
[12, 15, 25] * 2 → [24, 30, 50]means 12×2, 15×2, 25×2.
Loop vs NumPy, timed
time.perf_counter() is a stopwatch: read it before and after, and subtract. 1_000_000 is one million; underscores in a number are only for readability.
import time
import numpy as np
bills = list(range(1_000_000))
start = time.perf_counter()
new = [b * 1.1 for b in bills]
list_time = time.perf_counter() - start
arr = np.array(bills)
start = time.perf_counter()
new_arr = arr * 1.1
np_time = time.perf_counter() - start
print(f"list loop: {list_time:.4f} s")
print(f"NumPy: {np_time:.4f} s")
print(f"NumPy is about {list_time / np_time:.0f}x faster")
One real run:
list loop: 0.0857 s
NumPy: 0.0055 s
NumPy is about 15x faster
Two more runs on the same machine gave 22x and 16x. Your numbers will differ (they depend on your computer and change slightly every run), but NumPy wins every time, and the gap grows with heavier maths.
Why arrays beat lists
| Real life | NumPy idea | What it means |
|---|---|---|
| Train coaches coupled together | Contiguous memory | The numbers sit side by side in one unbroken block of memory. A list stores each value as a separate Python object somewhere in memory and only keeps a pointer (an address) to each one. |
| Every tray slot fits one egg | One dtype | Every element has the same type, so the computer never checks "is this a number or text?" per item. A list must check every item. |
| A machine vs doing it by hand | Loops in C | The loop still happens, inside NumPy, in code written in C, a compiled language (translated to machine code ahead of time), which runs far faster than Python stepping through items. |
You can see the tidy storage:
import numpy as np
week = np.array([320, 280, 410, 300, 350, 390, 450])
print(week.itemsize, "bytes per number")
print(week.nbytes, "bytes in total")
mixed = [320, "N/A", 4.5, True]
print([type(x).__name__ for x in mixed])
8 bytes per number
56 bytes in total
['int', 'str', 'float', 'bool']
- A byte is a small unit of computer memory (8 bits; enough for one English letter). A bit is a single 0 or 1.
itemsizeis the bytes per element;nbytesthe bytes for the whole array: 7 × 8 = 56, nothing wasted.- A list happily mixes an
int, astr, afloatand abool. Flexible, but that flexibility is what makes it slow for maths.
dtype: the array's one data type
An attribute is a value stored on an object, read with a dot and no brackets: a.dtype, a.shape. (A method is a function on an object, called with a dot and brackets: a.sum().)
The dtype (data type) is the one type every element shares.
import numpy as np
a = np.array([320, 280, 410])
b = np.array([1, 2.5])
c = np.array([320, "N/A", 410])
print(a.dtype)
print(b, b.dtype)
print(c, c.dtype)
int64
[1. 2.5] float64
['320' 'N/A' '410'] <U21
- int64: whole numbers stored in 64 bits (8 bytes) — that's where the 8 bytes came from.
- float64: decimal (floating-point) numbers in 64 bits. Mixing
1and2.5upcasts (upgrades) everything to float64, so1prints as1.. - <U21: Unicode text, up to 21 characters each. Unicode is the standard that gives every character in every language a number. One
"N/A"turned the whole array into text:320became the string'320'.
Maths on that text array fails:
import numpy as np
c = np.array([320, "N/A", 410])
print(c + 5)
Traceback (most recent call last):
File "main.py", line 3, in <module>
print(c + 5)
~~^~~
numpy._core._exceptions._UFuncNoLoopError: ufunc 'add' did not contain a loop with signature matching types (dtype('<U21'), dtype('int64')) -> None
In plain words: NumPy has no routine for adding a number to text. Always check .dtype after loading data: text where you expected numbers is the first sign of dirty data (cleaned properly in episodes 19-21).
You can also choose the dtype yourself:
import numpy as np
print(np.array([True, 2, 3.5]).dtype, np.array([1,2,3], dtype=float))
print(np.array([10, 20]).astype(float))
float64 [1. 2. 3.]
[10. 20.]
dtype=float sets the type when creating; .astype(float) makes a converted copy of an existing array.
shape, ndim and size
Ramesh has a second stall in Bandra. Two separate arrays would drift apart (like the parallel lists in episode 5). Better: one 2-D (two-dimensional) array — one row per stall, one column per day. A dimension (or axis) is a direction you can move along: along a row, or down a column.
- shape: how many elements along each dimension, as a tuple.
(7,)= 7 in one dimension;(2, 7)= 2 rows × 7 columns. - ndim: the number of dimensions.
- size: the total number of elements.
import numpy as np
week = np.array([320, 280, 410, 300, 350, 390, 450])
print(week.shape, week.ndim, week.size)
stalls = np.array([[320, 280, 410, 300, 350, 390, 450],
[210, 190, 260, 240, 230, 280, 300]])
print(stalls)
print(stalls.shape, stalls.ndim, stalls.size)
print(stalls.dtype)
(7,) 1 7
[[320 280 410 300 350 390 450]
[210 190 260 240 230 280 300]]
(2, 7) 2 14
int64
(7,) is a tuple with one number (the comma makes it a tuple). A 2-D array is built from a list of lists, one inner list per row. Every row must be the same length:
import numpy as np
bad = np.array([[320, 280, 410],
[210, 190]])
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
Inhomogeneous means "not all the same": the rows have different lengths.
Building arrays fast: arange, linspace, zeros, ones, full
Real-life examples: token numbers at a hospital counter (start, stop, step) → np.arange; the marks on a ruler, evenly spaced with both ends included → np.linspace; a blank register page, or one pre-filled with a target → np.zeros, np.ones, np.full.
import numpy as np
print(np.arange(5))
print(np.arange(1, 8))
print(np.arange(0, 50, 10))
print(np.arange(0, 1, 0.25))
print(np.linspace(0, 1, 5))
print(np.linspace(10, 20, 3))
[0 1 2 3 4]
[1 2 3 4 5 6 7]
[ 0 10 20 30 40]
[0. 0.25 0.5 0.75]
[0. 0.25 0.5 0.75 1. ]
[10. 15. 20.]
np.arange(start, stop, step): likerange(), but returns an array, and accepts decimal steps. The stop is excluded.np.linspace(start, stop, num)("linear space"):numevenly spaced points, both ends included, always float.- Rule of thumb: with
arangeyou choose the step; withlinspaceyou choose how many points. For decimal steps preferlinspace, since floating-point steps inarangecan give one point more or fewer than you expect.
import numpy as np
print(np.zeros(4))
print(np.ones(3, dtype=int))
print(np.full(7, 300))
print(np.zeros((2, 3)))
target = np.full(7, 300)
week = np.array([320, 280, 410, 300, 350, 390, 450])
print(week - target)
[0. 0. 0. 0.]
[1 1 1]
[300 300 300 300 300 300 300]
[[0. 0. 0.]
[0. 0. 0.]]
[ 20 -20 110 0 50 90 150]
np.zeros(shape): all 0, float64 by default.np.ones(shape): all 1.np.full(shape, value): allvalue.shapeis a number for 1-D, or a tuple like(2, 3)for 2-D.week - targetshows how far each day was from the 300-cup target, element by element.
Element-wise maths between two arrays
import numpy as np
cups = np.array([320, 280, 410])
price = np.array([12, 12, 15])
print(cups * price)
print(cups + cups)
print(cups > 300)
[3840 3360 6150]
[640 560 820]
[ True False True]
Elements pair up by position: first with first, second with second. Revenue (money taken in) per day is cups * price. A comparison gives a boolean array of True/False — the basis of filtering in episode 9.
Shapes must match:
import numpy as np
cups = np.array([320, 280, 410])
price = np.array([12, 12, 15, 15])
print(cups * price)
Traceback (most recent call last):
File "main.py", line 4, in <module>
print(cups * price)
~~~~~^~~~~~~
ValueError: operands could not be broadcast together with shapes (3,) (4,)
Operands are the values an operator works on (cups and price). The fourth price has no partner. Broadcasting is NumPy's rule for stretching a smaller array (like a single number) across a bigger one; that's why prices * 2 works. Its full rules come in episode 10.
Universal functions (ufuncs)
Real-life example: a calculator's √ button works on one number; imagine a √ button that works on a whole column at once. That's a universal function (ufunc): a NumPy function that runs element-wise over an entire array, in fast C. Even + and * on arrays call ufuncs (np.add, np.multiply).
import numpy as np
area = np.array([16.0, 25.0, 36.0])
print(np.sqrt(area))
growth = np.array([1.1234, 0.987, 1.05])
print(np.round(growth, 2))
print(np.maximum(np.array([320, 280, 410]), 300))
print(np.abs(np.array([20, -20, 110])))
print(type(np.sqrt))
[4. 5. 6.]
[1.12 0.99 1.05]
[320 300 410]
[ 20 20 110]
<class 'numpy.ufunc'>
| Function | Does | Example result |
|---|---|---|
np.sqrt |
square root of each element | [4. 5. 6.] |
np.round(a, 2) |
round each element to 2 decimals | [1.12 0.99 1.05] |
np.maximum(a, 300) |
the bigger of each element and 300 (a floor: a minimum allowed value) | [320 300 410] |
np.abs |
absolute value (drops the minus sign) | [ 20 20 110] |
np.add, np.multiply |
what + and * call |
Python's math.sqrt takes one number only, so it fails on an array:
TypeError: only 0-dimensional arrays can be converted to Python scalars
With arrays, use the np. version.
Mini project: two stalls' weekly revenue
import numpy as np
cups = np.array([[320, 280, 410, 300, 350, 390, 450],
[210, 190, 260, 240, 230, 280, 300]])
price = np.full((2, 7), 12)
revenue = cups * price
print(revenue)
new_price = np.round(price * 1.1)
print(new_price[0][:3])
new_revenue = cups * new_price
print("Old total:", revenue.sum())
print("New total:", new_revenue.sum())
print("Extra:", new_revenue.sum() - revenue.sum())
[[3840 3360 4920 3600 4200 4680 5400]
[2520 2280 3120 2880 2760 3360 3600]]
[13. 13. 13.]
Old total: 50520
New total: 54730.0
Extra: 4210.0
- 14 revenues at once, no loop.
12 * 1.1 = 13.2rounds to13. .sum()is a method that adds up every element.54730.0has.0becausenp.roundreturned float64 — the dtype lesson, live. (new_price[0][:3]peeks at the first 3 values of row 0; indexing 2-D arrays properly is episode 9.)- About Rs 4,210 extra a week, if sales stay the same after the price rise — a real analyst would check that next.
Common mistakes
| Code | Real error | Why | Fix |
|---|---|---|---|
[12, 15, 25] + 5 |
TypeError: can only concatenate list (not "int") to list |
For lists, + joins lists |
np.array([12, 15, 25]) + 5 |
[12, 15, 25] * 1.1 |
TypeError: can't multiply sequence by non-int of type 'float' |
For lists, * repeats |
np.array(prices) * 1.1 |
[12, 15, 25] * 2 expecting doubling |
(no error: [12, 15, 25, 12, 15, 25]) |
List repetition | Use an array |
np.array([320, "N/A", 410]) + 5 |
numpy._core._exceptions._UFuncNoLoopError: ufunc 'add' did not contain a loop with signature matching types (dtype('<U21'), dtype('int64')) -> None |
One string made the dtype text | Check .dtype; clean the data first |
cups (3,) * price (4,) |
ValueError: operands could not be broadcast together with shapes (3,) (4,) |
Element-wise needs matching shapes | Make the lengths match |
| Rows of different lengths | ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. ... |
Every row must be the same length | Fix the data, or fill gaps |
np.array(1, 2, 3) |
TypeError: array() takes from 1 to 2 positional arguments but 3 were given |
np.array takes one list |
np.array([1, 2, 3]) |
np.zeros(2, 3) |
TypeError: Cannot interpret '3' as a data type |
The shape must be one tuple | np.zeros((2, 3)) |
math.sqrt(array) |
TypeError: only 0-dimensional arrays can be converted to Python scalars |
math works on one number |
np.sqrt(array) |
Practice
1. Put a week of screen time in hours, [3.5, 5.2, 4.1, 2.0, 6.3, 4.8, 3.0], in an array and convert it to minutes in one line.
Solution
import numpy as np
screen = np.array([3.5, 5.2, 4.1, 2.0, 6.3, 4.8, 3.0])
print(screen * 60)
[210. 312. 246. 120. 378. 288. 180.]
2. Build 0 to 100 in steps of 5 with arange, and 11 evenly spaced points from 0 to 100 with linspace.
Solution
print(np.arange(0, 101, 5))
print(np.linspace(0, 100, 11))
[ 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85
90 95 100]
[ 0. 10. 20. 30. 40. 50. 60. 70. 80. 90. 100.]
The stop of arange is excluded, so use 101 to include 100. Long arrays wrap onto a second line when printed.
3. Time a list comprehension against NumPy for squaring 1,000,000 numbers.
Solution
import time
import numpy as np
nums = list(range(1_000_000))
start = time.perf_counter()
sq = [n ** 2 for n in nums]
t1 = time.perf_counter() - start
arr = np.array(nums)
start = time.perf_counter()
sq2 = arr ** 2
t2 = time.perf_counter() - start
print(f"loop {t1:.4f} s, NumPy {t2:.4f} s")
print(sq[:3], sq2[:3])
One real run:
loop 0.0846 s, NumPy 0.0041 s
[0, 1, 4] [0 1 4]
4. Convert Mumbai temperatures [31.5, 33.0, 29.8, 35.2] °C to °F (F = C × 9/5 + 32), then round.
Solution
temps_c = np.array([31.5, 33.0, 29.8, 35.2])
temps_f = temps_c * 9 / 5 + 32
print(temps_f)
print(np.round(temps_f))
[88.7 91.4 85.64 95.36]
[89. 91. 86. 95.]
5. Prices [120, 85, 240, 60] with discounts [0.10, 0.05, 0.20, 0.0]. Find the sale prices, then make sure nothing sells below Rs 80.
Solution
mrp = np.array([120, 85, 240, 60])
discount = np.array([0.10, 0.05, 0.20, 0.0])
print(mrp * (1 - discount))
print(np.maximum(mrp * (1 - discount) , 80))
[108. 80.75 192. 60. ]
[108. 80.75 192. 80. ]
6. Make a 3 × 4 array of integer zeros and print its shape, ndim and size.
Solution
grid = np.zeros((3, 4), dtype=int)
print(grid, grid.shape, grid.ndim, grid.size)
[[0 0 0 0]
[0 0 0 0]
[0 0 0 0]] (3, 4) 2 12
Quick quiz
import numpy as np
a = np.array([2, 4, 6])
print(a * 2, [2, 4, 6] * 2)
A) [4 8 12] [4, 8, 12] B) [ 4 8 12] [2, 4, 6, 2, 4, 6] C) [2 4 6 2 4 6] [4, 8, 12] D) TypeError
Answer
B. The real output is [ 4 8 12] [2, 4, 6, 2, 4, 6]. The array multiplies element-wise; the list repeats itself. (NumPy pads numbers with spaces so columns line up.)
Interview questions
- Why is NumPy faster than a Python list for numeric work? Contiguous memory, one fixed dtype (no per-item type checks), and loops that run in compiled C instead of the Python interpreter.
- What is vectorisation? Expressing an operation on a whole array at once, letting NumPy's compiled loop do the work instead of a Python
forloop. - What does
[1, 2] * 3give vsnp.array([1, 2]) * 3?[1, 2, 1, 2, 1, 2](repetition) vs[3 6](element-wise). - What happens if you create an array from
[1, 2.5, "x"]? Everything becomes a string dtype (<U32or similar); maths on it fails. - Difference between
np.arangeandnp.linspace?arangetakes a step and excludes the stop;linspacetakes a number of points and includes both ends. - What are
shape,ndimandsize? Elements per dimension (tuple), number of dimensions, total element count. - What is a ufunc? A universal function: a NumPy function that operates element-wise on arrays in C (
np.sqrt,np.add,np.maximum). - When does
operands could not be broadcast togetherappear? When element-wise maths is done on arrays whose shapes are incompatible, like(3,)and(4,).
Glossary
| Term | Meaning | Example |
|---|---|---|
| Library | A package of ready-made code you import | import numpy as np |
| NumPy | Numerical Python: the library for fast maths on arrays | np.array([1, 2]) |
as (import alias) |
A short nickname for an imported module | np for numpy |
| ndarray | NumPy's N-dimensional array: a grid of same-type values | <class 'numpy.ndarray'> |
| np.array | Creates an ndarray from a list (or list of lists) | np.array([12, 15, 25]) |
| Element | One value inside an array | 15 in [12 15 25] |
| Sequence | An ordered collection like a list or string | [12, 15, 25] |
| Concatenate | Join end to end | [1] + [2] → [1, 2] |
| Vectorisation | One operation on a whole array, no loop written by you | prices * 1.1 |
| Element-wise operation | Applied to each element separately, pairing by position | [1 2] * [3 4] → [3 8] |
| Floating-point approximation | How computers store most decimals: very close, not exact | 13.200000000000001 |
| time.perf_counter() | A precise stopwatch reading, in seconds | end - start |
| Contiguous memory | Values stored side by side in one unbroken block | an array's numbers |
| Pointer | A stored memory address of an object | what a list keeps per item |
| C / compiled language | A language translated to machine code ahead of time; fast | NumPy's loops |
| Bit / byte | A bit is one 0 or 1; a byte is 8 bits | itemsize = 8 bytes |
| itemsize / nbytes | Bytes per element / bytes for the whole array | 8, 56 |
| Attribute | A value on an object, read with a dot, no brackets | a.dtype |
| Method | A function on an object, called with a dot and brackets | a.sum() |
| dtype | The one data type all elements share | int64 |
| int64 | Whole numbers in 64 bits | 320 |
| float64 | Decimal numbers in 64 bits | 2.5 |
| Upcasting | Converting all values to a type that fits every one | [1, 2.5] → float64 |
Unicode / <U21 |
The standard numbering every character; <U21 = text up to 21 characters |
'N/A' |
| astype | Makes a converted copy with another dtype | a.astype(float) |
| Dimension / axis | A direction you can move along in an array | rows, columns |
| 2-D array | An array with rows and columns | (2, 7) |
| shape | Elements per dimension, as a tuple | (2, 7) |
| ndim | Number of dimensions | 2 |
| size | Total number of elements | 14 |
| Inhomogeneous | Not all the same (rows of different length) | [[1, 2], [3]] |
| np.arange | Like range() but an array; start, stop (excluded), step |
np.arange(0, 50, 10) |
| np.linspace | N evenly spaced points, both ends included | np.linspace(0, 1, 5) |
| np.zeros / np.ones | Arrays filled with 0 / 1 | np.zeros((2, 3)) |
| np.full | Array filled with a chosen value | np.full(7, 300) |
| Revenue | Money taken in: quantity × price | cups * price |
| Boolean array | An array of True/False from a comparison |
cups > 300 |
| Operand | A value an operator works on | cups in cups * price |
| Broadcasting | NumPy's rule for stretching a smaller array across a bigger one (episode 10) | prices * 2 |
| Universal function (ufunc) | A NumPy function applied element-wise in C | np.sqrt, np.add |
| Floor (minimum) | The lowest allowed value | np.maximum(a, 300) |
| Absolute value | A number without its minus sign | np.abs(-20) → 20 |
| Why arrays beat lists | One dtype, contiguous memory, loops in C | many times faster than a list loop |
Summary
- Lists can't do maths:
*repeats and+concatenates;* 1.1is aTypeError. - NumPy's ndarray holds one dtype in contiguous memory; vectorised, element-wise operations do the maths in one line with loops in C.
- dtype:
int64,float64, and one stray string turns the array into text (<U21) — check it after loading data. - shape, ndim, size describe an array; 2-D arrays have rows and columns of equal length.
- np.arange (step, stop excluded), np.linspace (count, both ends), np.zeros / ones / full build arrays fast.
- Element-wise maths needs matching shapes; ufuncs like
np.sqrt,np.round,np.maximumandnp.abswork on every element. - Next episode: NumPy 2: indexing, masks, reshaping and axes.