Text & documents

Numbers & maths

Data & formats

Security

Development & DevOps

Artificial Intelligence

Finance

Health & Wellness

Productivity

Games & Entertainment

Multimedia & design

Business

What it is and what it's for

Regression finds the curve that best fits a cloud of points, so you can predict the value y from an x. Each point is an observation (for example, study hours → grade); the fitted curve summarizes the trend and lets you estimate the y of an x you never measured. You see it live here: move points and the curve, the error and R² recompute instantly, all in your browser.

How to use it (step by step)

1. Drag any point to move it; the curve refits when you release.
2. Click on an empty spot of the chart to add a new point.
3. Move the Degree control to go from line to curve, and Regularization λ to smooth it.
4. Read the fit's MSE and R² below.
5. Buttons: Remove last deletes the last point, Clear deletes them all, and Reset returns to the starting example (10 points, degree 1, λ 0).

Linear regression (the line)

With Degree 1 the fit is a straight line y = a·x + b: classic linear regression. It's the simplest, most robust model: it captures the overall trend (whether y rises or falls as x grows, and with what slope) without being swayed by the noise of each point. Always start here; raise the degree only if the line clearly doesn't follow the shape of the data.

Least squares (how it fits)

The curve is chosen by least squares: it looks for the one that minimizes the sum of squared errors between each point and the curve (the residuals). They are squared so that errors above and below don't cancel out and so that large ones are penalized more. Internally it solves an exact linear system (normal equations), not by trial and error. Minimizing that error is equivalent to maximum likelihood (MLE) if the data noise is assumed to be Gaussian.

Polynomial degree and overfitting

The Degree (from 1 to 7) is how many bends the polynomial can make: degree 1 = line, 2 = parabola, and the higher the degree, the more wiggles. A high degree lowers the error on these points but usually gets worse: it overfits, i.e. it passes through almost every point chasing the noise and then generalizes poorly to new data. It's the bias-variance trade-off: too simple doesn't capture the shape, too complex memorizes the noise. Pick the lowest degree that follows the trend.

Residuals, MSE and R²

The vertical grey lines are the residuals: the distance from each point to the curve, i.e. the error the model makes at that point.

• MSE (loss) — mean squared error: the average of those squared residuals. Lower = better fit; it's exactly what the fit minimizes.
• R² — coefficient of determination: what fraction of the variation in y the model explains. It ranges from 0 to 1: 1 = the curve passes through every point, 0 = no better than always predicting the mean (and it can be negative if it fits worse than the mean itself).

Error on unseen data

MSE measures how well the curve learns its own points, and with more degree it always goes down. The error on unseen data measures what matters: how well it guesses a new one. It is computed by leaving each point out, fitting without it and seeing how far off it predicts it (leave-one-out cross-validation). With the example data: degree 1 → 0.36; degree 7 → 25.7, even though its MSE is lower. When one goes down and the other goes up, that is overfitting; raise λ and watch it rein it in.

The equation and the prediction

The top right shows the equation of the fitted curve: with degree 1, y = a·x + b. Those coefficients are everything the model has learned from the points; training is finding them, which is why they change as you drag.

The Predict at x control is using the model: pick an x and read the y the curve gives (the green point). If you take it outside the range of the data, you are extrapolating: with degree 1 the line stays sensible; with degree 7 the curve shoots off at the edges, which is the most direct way to see why a model that fits its data beautifully can predict terribly.

Regularization λ (ridge = MAP)

Regularization λ (from 0 to 10) fights overfitting without lowering the degree: it penalizes the polynomial's large weights (ridge), which smooths the curve and makes it less sensitive to noise. At λ = 0 it's pure least squares; as λ rises, the curve flattens and tends toward a straight line. It's equivalent to adding a Gaussian prior on the weights: in Bayesian terms, you move from MLE to MAP. The penalty does not affect the independent term (intercept).

Regression & MLE

Least-squares fit, overfitting and regularization (ridge = MAP)

Move the cursor over the chart to read its coordinates.Tap the chart or a point to read its coordinates.

Drag the points. Click empty space to add a point. The gray lines are the residuals (each point's error).

To move the view, drag with the right button or the pressed wheel (on a touch screen, press and hold, then drag). The wheel zooms in and out.

Drag the points. A tap on an empty spot adds a point. The grey lines are the residuals (the error at each point).

To move the view, press and hold for a moment, then drag. The buttons at the top zoom in, zoom out and fit.

y = 0.687·x + 1.658

Degree

Regularization λ

Predict at x

Predictionx = 5 → y = 5.094

MSE (loss)0.236

R²0.942

Error on unseen data0.363

Minimizing MSE = maximum likelihood with Gaussian noise (MLE). Raising the degree overfits noisy data; λ (ridge) is a Gaussian prior on the weights → MAP. It's the bias-variance trade-off.