Machine Learning Terms Cheat Sheet

What this glossary covers

These are the 20 terms you will run into as soon as you go one level deeper than "AI" — the vocabulary of building and training models. For general AI terms (prompt, token, hallucination), see the AI Vocabulary Cheat Sheet; for how the pieces fit into a workflow, see the Machine Learning Basics Reference.

Data terms

• Features — the input variables a model uses to make a prediction. For a house-price model: square footage, location, number of bedrooms.
• Labels — the correct answers attached to training examples. For that house-price model, the label is the actual sale price.
• Training set — the portion of your data the model learns from.
• Test set — data held back and never shown during training, used to measure how the model performs on examples it has not memorized.
• Validation set — a third slice, used during development to compare settings and model versions without touching the test set.
• Feature engineering — transforming raw data into more useful inputs, like turning a timestamp into "day of week." Often where projects are won or lost.

Training terms

• Loss function — a formula that scores how wrong the model's predictions are. Training is the process of driving this score down.
• Gradient descent — the standard method for reducing loss: repeatedly nudge the model's parameters in the direction that shrinks the error, like walking downhill in small steps.
• Learning rate — how big each of those steps is. Too large and training overshoots; too small and it crawls.
• Epoch — one full pass through the entire training set. Models typically train for many epochs.
• Batch — a small group of examples processed together in one update step, rather than the whole dataset at once.
• Weights — the individual learned numbers inside a model (its parameters). Training means adjusting the weights.
• Hyperparameters — settings you choose before training, such as learning rate or tree depth. The model learns its weights; you pick its hyperparameters.

Model behavior terms

• Overfitting — the model memorizes the training data, quirks and all, then fails on new data. Great training scores, poor real-world results.
• Underfitting — the model is too simple to capture the real pattern, so it performs poorly everywhere.
• Generalization — the whole point: performing well on new, unseen data, not just the examples used for training.
• Regularization — techniques that deliberately restrain a model during training to fight overfitting.
• Bias and variance — the classic tension. High bias means the model is too rigid (underfits); high variance means it is too sensitive to its particular training data (overfits). Good models balance the two.

Prediction terms

• Inference — using the trained model to make predictions on real inputs. Training happens once; inference happens every time the model is used.
• Confidence score — a number many models attach to a prediction indicating how sure they are. Useful, but not a guarantee — a model can be confidently wrong.

How to study these

Do not memorize the list in one sitting. Instead, read Machine Learning in Plain English and return here whenever a term stops you. Meeting each word inside a real explanation is what makes it stick.