What is a Neural Network? | AI Glossary

Start here

Neural Network in plain language

The name is inspired by biological neurons, but a neural network is not a literal digital brain. A basic unit combines weighted inputs and a bias, then applies an activation function. Nonlinear activations let networks represent relationships beyond a single linear transformation.

Network size and training requirements vary substantially. A small teaching network does not require the same resources as a large language model, and neural-network training does not always depend on manually labelled data.

How it works

Represent the input
Convert the task’s information into numerical inputs with a consistent meaning and scale.
Compute through the layers
Apply learned weights, biases and activation functions. The final layer produces the task’s output, such as a score or predicted value.
Adjust during training
Calculate a loss and use gradient-based updates to improve parameters. Evaluate on separate data to check whether the learned pattern generalises.

Illustrative example · Fictional data

One unit, with numbers you can check

Input: A fictional teaching unit receives x₁ = 2 and x₂ = 1. Its weights are 0.5 and −1, and its bias is 0.2.
Process: Compute z = (0.5 × 2) + (−1 × 1) + 0.2 = 0.2. Apply ReLU, which returns max(0, z).
Expected output: The unit outputs 0.2. If x₂ changes to 2, z becomes −0.8 and the ReLU output becomes 0.
How to check it: These are chosen teaching parameters, not a trained predictor. The output is not automatically a probability, and the calculation alone says nothing about real-world accuracy.

Where it helps

Learning representations for text, images or audio.
Modelling nonlinear relationships where simpler baselines are insufficient.
Producing features that support classification, prediction or generation.

Training, inference and architecture

Architecture: The arrangement of layers and operations: the model’s structure.
Training: The procedure that adjusts parameters using data and an objective.
Inference: Applying the configured model to an input. This does not normally update its weights.

Common pitfalls

A larger network can overfit or cost more without improving the intended task.
A diagram of connected circles does not explain what the model has learned.
A hidden-layer activation is not necessarily a human-readable concept.

Put the idea to work

Try it yourself

Repeat the unit calculation with x₁ = 0 and x₂ = 1. Then change only the bias to 2. Record which changes produce a zero ReLU output and why.

Common questions

Is a neural network the same as deep learning?

Deep learning uses multilayer neural networks. Neural networks also include small, shallow architectures. Deep learning typically uses many layers to learn features directly from raw data such as text, images or audio. A shallow network may have only one hidden layer.

Does every neural network need a GPU?

No. Required hardware depends on model size, data and workload. Small networks can run and train on a CPU. GPUs mainly speed up training and serving large models by running many calculations in parallel. Many classification or forecasting models train comfortably on a laptop.

Is every output a probability?

No. Its interpretation depends on the output transformation and task. A raw activation or score should not be called a probability without justification. Even probability-like outputs, such as softmax scores, can be poorly calibrated. Check calibration on held-out data before using scores as confidence levels in decisions.

Your next step