0.1 Functions and graphs: a model is a function with knobs
A function is a machine that turns an input number into an output number, and a machine learning model is a function whose knobs get turned until its graph runs close to the data.
01A function is a machine
A function is a machine with one fixed rule: a number goes in, the rule is applied, and one number comes out. In the viewport the cyan dot on the horizontal axis is the input. It slides from left to right, the cyan guide carries it to the green line, and the green marker on the vertical axis shows the output.
The rule in the viewport is multiply by 2, then add 1. Feed it 2: , then . Feed it : , then . The same input always gives the same output. That is the one promise every function makes.
Stop the sweep and feed the machine one input of your choice: pause at .
Khan Academy: Algebra, Calculus and Statistics, Khan Academy, Algebra 1: Functions
02Pairs of numbers draw a graph
Every time you feed the machine an input, you get a pair: the input and its output. Read the pair as directions: go across, then up, and put a dot there. In the viewport, each time the sweep passes a whole number it leaves a green dot labelled with its pair.
For : , , , and . Those five pairs are the five green dots. Inputs in between fill the gaps: gives , halfway between the dots at and . Every input at once makes the bright line, called the graph of .
Pause the sweep with the input at , between two of the dots.
03The straight line: slope and intercept
Every straight line has two knobs. The slope says how steep it is: how far up you go for each step to the right. The intercept says where the line crosses the vertical axis. The viewport is set to and . The amber triangle goes 1 across and 0.5 up, and the amber ring sits at height 1.5 on the vertical axis.
To find the slope from two points, count. From to you move 2 across, the run, and 1 up, the rise. The slope is rise over run: . The intercept then follows from either point:
- 1
The line passes through (1, 2): put x = 1 and f(x) = 2 into f(x) = 0.5x + b.
Make the line fall: set and keep .
Khan Academy: Algebra, Calculus and Statistics, Khan Academy, Algebra 1: Linear equations and graphs
04Other curve families and their knobs
A straight line is one family of functions. Change the kind of rule and you get a different family, each with its own knobs. Switch families with the Family control and the curve morphs from one shape to the next.
Quadratic. The new knob multiplies and bends the line into a U. The viewport has , and , so and as well: squaring forgets the sign, which is why the U is symmetric. A negative turns the U upside down.
Exponential, : each step of 1 to the right multiplies the output by the same factor, , so it grows faster and faster when and fades toward 0 when . Sigmoid, : an S-shaped curve squeezed between 0 and 1, the two dashed lines. Here sets how steep the S is and slides it sideways. The same knob names do different jobs in different families.
Flip and shift the U: set , and .
Switch to the exponential with and .
Switch to the sigmoid with and .
05Data, gaps and the total squared error
Now the job turns around. Instead of a rule you are given data: twelve measured pairs, the cyan dots. You want a line that passes close to all of them. For each dot, the rose stick is the vertical gap between the dot and the line, how far the line's output misses the real output at that input.
The viewport starts with a flat line, and . The dot at has , but the line outputs , so the gap is . Squaring gives , the area of the rose square drawn on that stick. While the scene plays it visits the dots one by one and adds up their squares. The total is .
Improve the line by hand: set and keep .
Mathematics for Machine Learning, Deisenroth, Faisal & Ong, Ch. 9.1 and 9.2
06Fitting: turn the knobs to shrink the error
Fitting means turning the knobs until the error is as small as it can get. You did it by hand in the last step. With Auto-fit on, the scene does it for you: many times a second it nudges and a little in whichever direction makes smaller, and you can watch the line swing into place while the squares melt.
It starts from a bad guess, and , where . Within about two seconds the slope is close to right; then the line slides down and settles at , and . No straight line scores lower on these twelve points. Your hand-tuned line from the last step was close, at .
Pick the wrong family: switch to the sigmoid, starting from and , with auto-fit on.
Give the model a spare knob: switch to the quadratic with , and , auto-fit on.
Mathematics for Machine Learning, Deisenroth, Faisal & Ong, Ch. 9.2
Questions learners ask
- What is a function in maths, in simple terms?
- A function is a rule that takes an input number and gives back exactly one output number. You can picture it as a machine: put 2 into the rule 'multiply by 2, then add 1' and 5 comes out, every time. Written as a formula this is f(x) = 2x + 1, and f(2) = 5 means the output for the input 2. Plotting every input against its output draws the function's graph.
- What do the slope and the intercept mean in y = mx + b?
- The slope m is how steep the line is: how far it goes up for each step of 1 to the right, found from any two points as rise divided by run. A negative slope means the line goes down. The intercept b is the height where the line crosses the vertical axis, which is the output when x is 0. Changing b moves the whole line up or down; changing m tilts it around that crossing point.
- Why is a machine learning model called a function with parameters?
- A model takes an input, such as a picture or a sentence or a list of measurements, and returns an output, such as a label or a number, so it is a function. Its rule contains adjustable numbers called parameters, like the slope and intercept of a line. Training means turning those numbers until the model's outputs are close to the right answers on example data, usually by making a total or average squared error as small as possible.
Sources
- Essence of Linear Algebra, 3Blue1Brown (Grant Sanderson)
- Khan Academy: Algebra, Calculus and Statistics, Khan Academy
- Mathematics for Machine Learning, Deisenroth, Faisal & Ong